Report  Documentation  Page 


Form  Approved 
0MB  No.  0704-0188 


Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  0MB  control  number. 


1.  REPORT  DATE 

FEB  2010 


2.  REPORT  TYPE 


3.  DATES  COVERED 

00-01-2010  to  00-02-2010 


4.  TITLE  AND  SUBTITLE 

CrossTalk:  The  Journal  of  Defense  Software  Engineering.  Volume  23, 
No.  1,  Jan/Eeb  2010 

6.  AUTHOR(S) 


7.  PEREORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

CrossTalk, 517  SMXS/MXDEA,6022  Eir  Ave,Hill  AEB,UT, 84056-5820 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 


5a.  CONTRACT  NUMBER 


5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

5d.  PROJECT  NUMBER 


5e.  TASK  NUMBER 


5f.  WORK  UNIT  NUMBER 

8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 


10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 


12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 


13.  SUPPLEMENTARY  NOTES 


14.  ABSTRACT 


15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF: 


a.  REPORT 

unclassified 


b.  ABSTRACT 

unclassified 


c.  THIS  PAGE 

unclassified 


17.  LIMITATION  OF 

18.  NUMBER 

ABSTRACT 

OE  PAGES 

Same  as 

40 

Report  (SAR) 

19a.  NAME  OE 
RESPONSIBLE  PERSON 


Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


Policies, 


and  Updates 


4  Announcing  Crosstalk’s  Co-Sponsor  Team  for  20 1 0 

Meet  Crosstalk’s  2010  co-sponsors. 


5  The  20 1 0  Crosstalk  Editorial  Board 

Crosstalk  presents  our  valued  board  of  article  reviewers  for  2010. 


^  An  Integrated  Framework  for  Performance  Excellence 

Find  out  how  a  CMMI-based  framework — utilizing  Lean  Thinking, 
Six  Sigma,  and  the  Information  Technology  Infrastructure  Library — 
can  be  a  highly  successful  approach  for  performance  improvement. 
bj  J^frej  L.  Dutton 


1  ^  Process  Improvement  for  All:  What  to  Expect  from 
A  U  CMMI  Version  1.3 

This  article  discusses  the  forthcoming  CMMI  Product  Suite  Version  1.3: 
rationale  for  the  update,  its  development  schedule,  and  its  improvements 
to  the  three  models  and  SCAMPI  appraisal. 
by  Mike  Phillips  and  Sandy  Shrum 


Scope  Management:  1 2  Steps  for  ICT  Program 
Recovery 

This  article  focuses  on  the  scope  management  process  northernSCOPE, 
a  means  of  leveraging  and  augmenting  professional  project  management 
on  information  and  communications  technology  projects. 
by  Carol  Dekkers  and  Pekka  Forselius 
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Stealth  CPI:  Managing  Work  Products  to  Achieve 
Continuous  Process  Improvement 

The  benefits  of  CPI  during  implementation  have  been  minimal,  but 
Warren  and  Abler’s  method  may  change  that. 
by  Ron  Abler  and  Ted  Warren 


CMMI,  Swiss  Cheese,  and  Pareto 

Corbin’s  case  study  of  a  CMMI  appraisal  preparation — including  Alan 
Lakein’s  “Swiss  Cbeese  Method”  and  the  Pareto  Principle — shows  a  way 
other  organizations  can  successfully  get  ready. 
by  Darrell  Corbin 


orum 


Love  and  Marriage:  CMMI  and  Agile  Need  Each  Other 

This  article  shows  that  when  working  together,  CMMI  and  Agile  can 
complete  each  others’  capabilities  and  lead  to  fast,  affordable,  visible, 
and  long-term  benefits. 
by  Hillel  Glaser 
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From  the  Sponsor 


Is  There  Progress  in  Process? 


CMMI:  Getting  a  Handk  on  ¥rocess\s,  easier  said  than  done.  The  constant 
barrage  of  variables  interacting  with  the  human  component  makes 
following  a  repeatable  process  model  a  challenge  in  any  industry — but 
extremely  difficult  in  software.  Building  assembly  line-type  products 
makes  process  easier,  but  the  defense  software  industry  doesn’t  typically 
make  assembly  line  software.  In  fact,  our  jobs  are  anything  but  repeatable 
as  we  strive  to  meet  the  challenges  of  an  ever-changing  enemy  with  its 
own  evolving  technological  advances,  cyberterrotism  strategies,  and  constant 
countermeasures  taken  to  defeat  our  latest  technology. 

So  why  put  forth  the  effort  to  follow  a  process  model  when  the  products 
change  so  rapidly?  The  reason  is  simple:  There  are  huge  benefits  accompanying 
process.  Our  software  team,  the  309th  Software  Maintenance  Group  (SMXG)  at  HiU  AFB,  has 
teamed  in  an  enterprise  fashion  with  two  other  AFB  SMXGs — the  76th  at  Tinker  and  the  402nd 
at  Robins — committing  to  process  improvement  in  an  effort  to  further  progress  down  the 
process  road  on  which  we  embarked  many  years  ago.  We  continue  because  past  efforts  have  sig¬ 
nificantly  reduced  defects  and  increased  productivity.  Process  can  do  that  in  a  way  no  other 
process  improvement  effort  can. 

Process  should  not  be  misconstrued  as  a  substitute  for  people,  individuality,  creativity,  and 
ingenuity — essential  elements  that  are  collectively  known  as  agility.  These  are  unique  factors  that 
must  harmoniously  remain  inside  process,  enhancing  within  the  nonrestrictive  parameters  of  a 
good  process  model.  After  all,  it  is  only  a  model. 

Fve  heard  the  many  arguments  made  in  defense  of  agility  versus  process.  This  issue  of 
Crosstalk  wiU  explore  the  virtues  of  both,  also  discussing  the  transformation  and  incorpo¬ 
ration  of  process  with  other  concepts,  ideas,  and  process  improvement  efforts. 

We  begin  with  Jeffrey  L.  Dutton’s  An  Integrated  Framework  for  Performance  Excellence,  which 
addresses  emerging  value  from  CMMI  and  the  associated  principles  for  implementation.  Mike 
Phillips  and  Sandy  Shrum  continue  the  CMMI  topic  with  their  article  Process  Improvement  for  All: 
What  to  Expect  from  CMMI  Version  1.3.  They  detail  changes  to  CMMI  for  Development, 
Acquisition,  and  Services — as  well  to  appraisal  and  training  methods — found  in  the  new  ver¬ 
sion,  scheduled  to  arrive  in  late  2010. 

Anyone  battling  project  control  issues  wiU  want  to  read  Scope  Management:  12  Steps  for  ICP 
Program  Recovery,  where  Carol  Dekkers  and  Pekka  ForseUus  describe  a  process  designed  to  con¬ 
trol  project  parameters.  Ron  Abler  and  Ted  Warren  discuss  the  other  side  of  the  process  coin 
by  looking  at  how  advancements  in  organizations  occur  by  focusing  on  work  products — not  just 
processes — in  Stealth  CPI:  Managing  Work  Products  to  Achieve  Continuous  Process  Improvement.  DarreU 
Corbin’s  CMMI,  Swiss  Cheese,  and  Pareto  comes  to  us  from  an  appraiser’s  point  of  view,  sharing  a 
case  study  describing  creative  techniques  used  to  achieve  100  percent  CMMI  compUance  in  a 
virtual  environment. 

FinaUy,  in  this  issue’s  Open  Forum  section,  HiUel  Glazer  explains  the  puzzle  piece  matching 
relationship  he  discovered  between  process  and  non-process  in  Eove  and  Marriage:  CMMI  and 
Agile  Need  Each  Other. 

This  issue  offers  something  for  everyone,  regardless  of  what  side  of  the  aisle  you  sit  on  in 
the  process  debate.  Getting  a  handle  on  process  may  not  be  easy,  but  it’s  worth  it.  We  hope  that 
this  issue  assists  you  and  your  organization  in  building,  incorporating,  integrating,  optimizing, 
and  managing  your  process  efforts. 
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Policies,  News,  and  Updates 


Announcing  CrossTalk’s  Co-Sponsor  Team  for  2010 


Kasey  Thompson 
Crosstalk 


/  would  like  to  once  again  express  sincere  thanks  to  the  2009  CROSSTALK  co-sponsors.  Simply  put,  CrossTalk  would 
not  exist  without  them  and  their  generous financial  support.  As  Publisher,  I  receive  countless  kudos — via  e-mail  and  the  phone — 
expressing  appreciation  for  an  article  or  issue  focus  that  contributed  to  individual  or  organiT(ational  success.  These  compliments 
really  belong  to  the  co-sponsors,  who  spark  countless  themes  and  help  bring  us  the  best  authors  in  defense  software  engineering. 
Tikewise,  it  is  my  pleasure  to  introduce  CrossTalk’j  2010  co-sponsor  team  and  offer  profound  gratitude  for  their  contin¬ 
ued  support  and  commitment  to  this  journal  I  know  firsthand  of  their  vision,  caring,  and  dedication  to  their  industry  and  it  is 
manifested  through  support  of  CROSSTALK.  Each  co-sponsor  and  their  organisation  will  assist  our  staff  by  lending  us  their 
inexhaustible  experience  in  engineering,  systems,  security,  acquisition,  tools,  processes,  models,  infrastructure,  people,  and  (of 
course)  software.  Co-sponsor  team  members  are  identified  in  this  section  with  a  description  of  their  organisation.  Please  look  for 
their  contributions  each  month  in  our  From  the  Sponsor  column,  found  on  page  3.  Their  organisations  will  also  be  highlighted 
on  the  back  cover  of  each  issue  of  CROSSTALK. 


Stephen  P.  Welby,  Office  of  the  Under  Secretary 
of  Defense  (AT&L)  -  Director,  Systems 
Engineering 

The  Systems  Engineering  directorate  is  responsi¬ 
ble  for  all  matters  relating  to  DoD  major  pro¬ 
gram  acquisition  systems  engineering,  software 
engineering,  system  assurance,  and  system  of 
systems  engineering.  The  directorate  includes  the  offices  of 
Major  Program  Support,  Mission  Assurance,  and  System 
Analysis,  with  responsibilities  in  acquisition  program  support  and 
oversight,  systems  engineering  policy  and  guidance,  human  capi¬ 
tal,  software  acquisition  management  and  engineering,  system 
integration,  and  government-industry  collaboration.  The  direc¬ 
torate’s  2010  focus  areas  include  workforce  development,  early 
systems  engineering,  simplifying  defense  acquisition  guidance, 
and  promoting  best  systems  engineering  practices  to  reduce  risk. 
See  <www.  acq.osd.mil/sse>  for  more  information. 


Joe  Jarzombek,  Department  of  Homeland 
Security  (DHS)  -  Director  of  Software 
Assurance  (SwA) 

The  DHS  National  Cyber  Security  Division 
serves  as  a  focal  point  for  SwA,  facilitating 
national  public-private  efforts  to  promulgate 
best  practices  and  methodologies  that  promote 
integrity,  security,  and  reliability  in  software  development  and 
acquisition.  Collaborative  efforts  of  the  SwA  community  have 
produced  several  publicly  available  online  resources.  For  more 
information,  see  the  Build  Security  In  Web  site  <https:/ /build 
securityin.us-cert.gov>  and  the  SwA  Community  Resources  and 
Information  Clearinghouse  <https://buildsecurityin.us-cert.gov 
/swa>.  Both  provide  coverage  of  topics  relevant  to  the  broader 
stakeholder  community. 


I  Joan  Johnson,  Naval  Air  Systems  Command 
I  (NAVAIR),  Systems  Engineering  Department - 
Director,  Software  Engineering 

I  NAVAIR  has  three  Strategic  Priorities  through 
which  it  produces  tangible,  external  results  for  the 
Sailor  and  the  Marine.  First  are  its  People  that  we 
I  develop  and  provide  the  tools,  infrastructure,  and 
processes  needed  to  do  their  work  effectively.  Next  is  Current 
Readiness  that  delivers  NAVAL  aviation  units  ready  for  tasking 
with  the  right  capability,  at  the  right  time,  and  the  right  cost. 
Finally  is  Future  Capability  in  the  delivery  of  new  aircraft, 
weapons,  and  systems  on  time  and  within  budget  that  meets  Fleet 
needs  and  provides  a  technological  edge  over  our  adversaries.  See 
<www.navair.navy.  mil>  for  more  information. 
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I  Karl  Rogers,  309  Software  Maintenance  Group 
(SMXG)  Acting  Director 

The  309th  SMXG  at  the  Ogden- Air  Logistics 
Center  is  a  recognized  world  leader  in  cradle-to- 
grave  systems  support,  encompassing  hardware 
engineering,  software  engineering,  systems  engi- 
I  neeting,  data  management,  consulting,  and  much 
more.  Their  accreditations  also  include  AS  9100  and  ISO  9000. 
See  <www.309SMXG.hiIl.af.mil>  for  more  information. 


Want  to  Become  a  Co-Sponsor? 

Crosstalk  co-sponsors  enjoy  many  benefits  such  as  inclu¬ 
sion  of  a  page-long  co-sponsor’s  note,  placement  of  their  orga¬ 
nization’s  logo  on  the  back  cover  for  six  issues,  placement  of  the 
Director’s  name  and  organization  on  each  issues’  masthead,  spe¬ 
cial  sponsorship  references  in  various  issues,  the  ability  to  pro¬ 
vide  authors  from  within  their  community  in  regard  to  their 
sponsored  issue,  and  online  placement  on  the  CROSSTALK 
Web  site. 

CrossTalk  co-sponsors  are  also  invited  each  year  to  pro¬ 
vide  direction  for  future  themes  and  feedback  from  the  software 
defense  community  at  large.  Co-sponsors  are  also  invited  to  par¬ 
ticipate  in  an  annual  meeting  held  during  the  Systems  and 
Software  Technology  Conference  to  discuss  emerging  needs, 
trends,  difficulties,  and  oppormnities  that  CROSSTALK  may 
address  in  an  effort  to  best  serve  its  readers. 

CrossTalk  welcomes  queries  regarding  potential  spon¬ 
sorship  throughout  the  year.  For  more  information  about  becom¬ 
ing  a  CrossTalk  co-sponsor,  please  contact  Kasey 
Thompson  at  (801)  586-1037  or  <kasey.thompson@hiIl.a£mil>. 
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Crosstalk  proudly  presents  the  2010  CROSSTALK  Editorial  Board.  Each  article  submitted  to  CROSSTALK  is  reviewed 
ly  two  technical  reviewers  from  the  following  list  Their  insights  improve  the  readability  and  usefulness  of  the  articles  that  we  pub¬ 
lish.  We  give  a  very  special  thanks  to  all  those  participating  in  our  2010  CROSSTALK  Editorial  Board. 


Wayne  Abba 
COL  Ken  Alford,  Ph.D. 

Bruce  AUgood 
Greg  Anderson 
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Dr.  Max  H.  Miller 
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DougJ.  Parsons 
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Gary  A.  Petersen 
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CMMI:  Getting  a  Handle  on  Process 


An  Integrated  Framework  for  Performance  Excellence 

Jeffrey  L.  Dutton 
Jacobs  Technology,  Inc. 

After  10  years  of  utilising  and  consulting  customers  in  CMMP,  the  author  uses  his  experiences  to  outline  three  “driving 
principles”  of  performance  improvement  focus  on  business  issues  / performance  goals,  involved  leadership! process  ownership, 
and  rapid  improvements);  compare  four  improvement  approaches  (CMMI,  Tean,  Six  Sigma,  and  the  Information  Technology 
Infrastructure  Tibrarj  fTILJ);  and  show  how  an  integrated  CMMI-based  framework  has  become  a  highly  successful 
approach  for  improvement. 


My  company’s  journey  with  CMMI 
began  as  a  member  of  the  Version  1.0 
Product  Team.  Since  supporting  that  effort 
in  1999  and  2000,  we  have  consulted  with 
customers  who  have  adopted  CMMI,  have 
adopted  it  ourselves  on  several  occasions, 
and  have  become  a  partner  with  the  SEI  for 
Appraisal  Services  and  Training.  We  have 
also  found  ourselves  part  of  a  growing 
community  that  continues  to  discover  the 
emerging  power  and  value  of  the  CMMI 
model  suite.  The  driving  principles  that 
have  emerged  from  this  body  of  work  are; 

•  An  improvement  effort  must  be 
focused  on  achieving  real  business  goals 
and  helping  the  organization  execute  its 
business  strategy. 

•  If  real  results  are  needed  in  a  meaning¬ 
ful  time  frame,  leadership  must  be 
involved  in  a  direct  and  real  way,  and 
process  doers  should  own  their  own 
processes. 

•  To  be  of  significant  value,  improve¬ 
ments  must  be  accomplished  at  the  speed 
of  business-,  often  in  weeks  or  months, 
rarely  in  years. 

We  were  driven  by  customer  needs  to 
attempt  integration  of  the  CMMI  frame¬ 
work  with  other  improvement  approaches. 
We  have  now,  along  with  our  customers, 
integrated  Lean  Thinking,  the  ITIL  frame¬ 
work,  and  Six  Sigma  mechanisms  into  the 
CMMI  framework.  Surprisingly,  we  found 
that  CMMI  models  consistently  provided  a 
synergistic  integrating  framework  for  these 
other  approaches,  resulting  in  CMMI-based 
performance  improvement  capabilities  that 
far  outreached  any  one  of  these  approaches 
alone. 

It  is  worth  mentioning  that  all  of  our 
initial  efforts  were  in  support  of  customer 
and  internal  software  development  organi¬ 
zations.  The  lessons  learned  in  this  article 
are  all  directly  applicable  to  smaU-to-large 
software  organizations.  Our  first  integration 

SCAMPI  is  a  service  mark  of  Carnegie  Mellon 
University. 

(g)  CMMI  is  registered  in  the  US.  Patent  and  Trademark 
Office  by  Carnegie  Mellon  University. 


of  Lean  Thinking  was  accomplished  by 
integrating  the  Lean  software  development 
constructs  [1]  into  the  CMMI  for  Develop¬ 
ment  (CMMI-DEV)  framework.  The  sur¬ 
prising  results  (discussed  in  this  article) 
encouraged  us  to  approach  the  integration 
of  other  improvement  approaches  with  a 
more  positive  and  hopeful  view.  These  ideas 
were  initially  briefed  by  the  author  (in  an 
acquisition  context)  to  the  DoD-sponsored 
Software  Acquisition  Fall  Workshop  (2007). 

In  this  article,  I  will  first  discuss  the 
three  driving  principles  of  performance 
improvement.  A  brief  comparative  discus¬ 
sion  of  the  four  improvement  approaches 
follows,  and  I  will  conclude  with  a  discus¬ 
sion  of  the  integrated  CMMI-based  frame¬ 
work. 

The  Driving  Principles 
Principle  #1:  Focus  on  Business 
Issues  and  Performance  Goals 

An  improvement  effort  without  focus  on 
performance  goals  or  the  resolution  of 
business  issues  resembles  a  missile  without 
a  guidance  system:  You’re  pretty  sure  it  will 
land,  but  you’re  just  not  quite  sure  where. 

The  performance  and/or  quality  goals 
should  be  of  real  business  importance  to 
the  organization’s  or  project’s  leadership, 
and  completely  in  line  with  its  business 
strategy.  Too  often,  improvements  become 
overly  oriented  on  the  process,  as  though 
the  process  were  an  end-goal.  Focus  on 
compliance  with  a  model  (Uke  the  CMMI) 
without  improving  performance  and/or 
work  product  quality  can  have  a  devastating 
effect  on  an  organization. 

All  improvement  models  and  approach¬ 
es  (including  CMMI,  Lean  Thinking,  Six 
Sigma,  and  ITIL)  can  be  implemented 
badly — that  is,  for  their  own  sake.  There  is 
no  hof  ff-ail  of  CMMI  achievement  (includ¬ 
ing  Maturity  Level  5)  that  will  guarantee  that 
an  organization  will  perform  better  unless  it 
consciously  sets  out  to  do  so  from  the 
beginning.  Level  3  organizations  may  not 
perform  any  better  than  Level  1  organiza¬ 


tions — and  some  actually  perform  worse! 

Figure  1  reflects  10  years  of  anecdotal 
evidence  of  the  relationship  between  focus 
(the  independent  variable)  and  cost  or  value 
(dependent  variables)  in  various  improve¬ 
ment  efforts. 

The  solid  line  indicates  how  business 
value  increases  dramatically  as  focus  on  real 
business  performance  goals  is  increased. 
The  reasons  for  this  improvement  in  value 
are  many,  but  perhaps  most  important  is 
that  when  the  organization  is  focused  on 
specific  performance/quality  goals,  CMMI 
practices  and  informative  components  can 
easily  be  implemented  in  the  context  of  the 
organization’s  business  case.  Practices  can 
be  implemented  as  is,  or  alternatives  written 
that  respond  better  to  the  business  case. 
Informative  components  can  more  readily 
be  sifted  for  implementation,  which  turns 
out  to  be  incredibly  helpful. 

Notice  that  very  unfocused  efforts  (near 
the  zero  abscissa  value)  can  and  have  actual¬ 
ly  resulted  in  improvement  efforts  that  have 
negative  value  to  an  organization. 

At  the  same  time,  cost  can  be  expected 
to  decrease  (as  indicated  by  the  dashed  line) 
to  an  optimum  least  cost.  The  primary  rea¬ 
son  for  this  is  that  a  sharper  focus  on  busi¬ 
ness  context  drastically  reduces  the  rework 
associated  with  model  implementations  that 
are  not  helpful  to  achieving  the  perfor¬ 
mance/quality  goals  of  the  organization.  In 
one  of  our  most  recent  improvement 
efforts,  costs  were  reduced  to  one-fourth  of 
that  for  previous,  similarly  scoped  efforts. 

Principle  #2;  Involved  Leadership  and 
Process  Ownership  by  Process  “Doers” 

Process  improvement  literature  discusses 
the  role  of  management  as  an  enabler — a 
function  that  provides  resources,  allows  the 
process  group  to  do  its  business,  and  spon¬ 
sors  the  process  improvement  activity.  This 
approach  is  fine  if  slow  progress,  inefficient 
and  ineffective  processes,  and  reluctant  buy- 
in  are  the  goals. 

Leadership  focused  on  performance 
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improvement  is  much  more  engaged,  pro¬ 
viding  the  business  goals  and  strategy  and 
the  realization  that  performance  improve¬ 
ment  efforts  are  exactiy  about  making  the 
business  succeed.  Leadership  is  directiy  and 
proactively  involved  in  the  improvement 
effort,  playing  an  important  role  on  a  week¬ 
ly  or  even  daily  basis.  Leadership  ensures 
everyone  in  the  organization  knows,  by 
example,  that  performance  improvement  is 
part  of  the  job,  not  separate  from  it. 

Perhaps  the  most  common  mistake 
organizations  can  make  is  to  assign  the 
development  of  processes  to  a  process  group 
that  is  not  made  up  of  the  process  doers. 
Even  if  the  group  is  extraordinarily  good  at 
eliciting  the  process  requirements  from  the 
actual  users  of  the  processes,  its  products 
(the  organization’s  processes)  will  be  resist¬ 
ed  or  even  thwarted.  The  literature  is  full  of 
cautions,  analyses,  and  workarounds  for  the 
push  back  or  resistance  from  process  doers 
when  the  process  group  deploys  a  new  or 
changed  process.  There  are  discussions  of 
what  to  do  with  heroes  when  they  cause  trou¬ 
ble  (!),  and  advice  on  how  to  deal  with  resis¬ 
tance — even  advice  on  how  to  mandate  com- 
plianceX 

Principle  #3:  Improvements  Should  Be 
Made  at  the  Speed  of  Business 

The  speed  of  business  today  is  driven  by 
rapid  changes  in  markets  and  in  technolo¬ 
gies.  Few  organizations  have  the  luxury  of 
two  or  three  years  to  make  meaningful 
improvements  to  their  performance.  Time 
frames  of  a  month  or  a  year  are  more  real¬ 
istic  and,  most  importantly,  more  responsive 
to  the  needs  of  the  organization. 

As  an  academic  exercise,  it  is  recognized 
that  improvement  velocity  has  both  speed 
and  direction.  Direction  can  be  thought  of 
as  focus,  as  discussed  earlier.  Speed,  of 
course,  has  to  do  with  how  fast  the 
improvement  effort  produces  artifacts  and 
changes  in  performance  or  work  product 
quality. 

Velocity  =  Speed  of  improvement  in  a 
focused  direction. 

We  studied  the  factors  associated  with 
the  institutionalization  of  a  process  in  an 
organization,  and  came  to  the  conclusion 
that  time  was  not  a  significant  attribute.  Of 
course,  there  exists  a  lower  bound  on  the 
time  to  implement  and  institutionalize 
processes  in  an  organization,  but  we  believe 
there  is  no  standard  time  (e.g,  two  years  to 
Maturity  Level  2,  four  years  to  Maturity 
Level  3,  etc.).  If  the  organization  is  focused 
on  achieving  important  performance/quali- 
ty  goals,  and  on  simultaneously  interpreting 
and  assuring  model  compliance,  success  is 


more  a  matter  of  achieving  the  organiza¬ 
tional  goals  while  working  through  the 
details  rather  than  of  the  passage  of  time. 

For  example,  in  the  case  where  the  cost 
of  an  improvement  effort  was  reduced  to 
one-fourth  that  of  similar  efforts,  time  was 
reduced  by  77  percent  (11  months  versus 
four  years) . 

I  now  win  provide  a  comparative  discus¬ 
sion  of  the  four  improvement  approaches 
we  have  used  most.  For  each  approach,  I 
will  provide  a  value  proposition,  a  look  at 
the  downsides,  and  end  with  a  brief  discus¬ 
sion  of  how  well  the  approach  integrates 
with  other  frameworks  for  improvement. 

Improvement  Approaches 

CMMi 

CMMI  is  basically  a  set  of  three  models — 
Development,  Services,  and  Acquisition — 
and  SCAMPI™.  The  models  and  appraisal 
methods  were  developed  by  integrated 
teams  composed  of  people  from  the  SEI, 
industry,  and  government  (1  was  an  industry 
member  of  the  CMMI  VI.O  product  team). 
CMMI  models  provide  best  practices  in  the 
three  domains  of  Development  (software, 
hardware,  and  systems).  Services  (any  kind, 
including  IT),  and  Acquisition.  The  models 
are  called  CMMI-DEV  [2],  CMMI  for 
Services  (CMMI-SVC)  [3],  and  CAIMI  for 
Acquisition. 

CMMI’s  value  lies  primarily  in  three 
areas; 

1.  The  three  sets  of  domain-specific  best 
practices. 

2.  Practices  that  enable  an  improvement 
infrastructure,  allowing  process  and  per¬ 
formance  improvements  across  five  dif¬ 
ferent  levels. 

3.  A  robust,  extensible  appraisal  method 
that  is  recognized  for  its  reliability  and 
credibility. 

The  specific  practices  contained  in  the 
CMMI-DEV  model  support  mature  sys- 

Figure  2:  Sampk  Process  Control  Chart 
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the  Focus  of  an  Improved  Fffort 

terns,  software  or  hardware  development, 
and  have  been  migrated  to  enable  both 
Lean  and  Agile  approaches. 

The  infrastructure  practices  provide 
guidance  for  developing  the  organi^tion  for 
improvement  as  well  as  necessary  supporting 
functions,  such  as  configuration  manage¬ 
ment,  quality  assurance,  and  metrics. 

SCAMPI  allows  for  responsive,  cost- 
efficient  assessments  of  ongoing  improve¬ 
ment  efforts,  and  is  extensible  to  include 
things  such  as  performance  goal  evaluation 
and  ITIL.  SCAMPI  allows  for  rapid  course 
corrections  in  improvement  efforts  as  well 
as  rapid  and  essential  learning  by  the  orga¬ 
nization,  and  provides  internationally  recog¬ 
nized  benchmarking. 

There  are  some  downsides  as  well. 
CMMI  does  not  contain  practices  or  guid¬ 
ance  for  setting  meaningful  business  or  per¬ 
formance  objectives,  or  for  formulating 
improvement  strategies  to  achieve  such 
objectives  (with  the  exception  of  the 
Strategic  Planning  process  area  within  the 
CMMI-SVC  model).  It  is  fair  to  note,  how- 
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Figure  3:  Overlapping  Attributes  of  Improvement  Approaches 


ever,  that  most  models  or  improvement 
approaches  omit  such  guidance. 

CMMI  models  are  integrahle  with  each 
other  in  what  is  called  multi-model  imple¬ 
mentations.  Examples  are  implementations 
of  the  CMMI-DEV  with  CMMI-SVC  for 
engineering  development  and  services  orga¬ 
nizations.  Our  experience  integrating  Lean 
Thinking,  Six  Sigma,  and  ITIL  into  CMMI- 
DEV  and  CMMI-SVC  models  has  revealed 
that  there  are  no  anti-patterns  or  other 
intractable  problems. 

Lean  Thinking 

Lean  is  much  more  than  building  efficient, 
streamlined  processes  and  the  potential 
reduction  of  resources  needed  to  perform  a 
process  or  service.  In  fact,  these  things  are 
secondary  to  the  point  of  Lean. 


Because  our  customers  were  beginning 
to  migrate  from  CMMI  to  a  Lean  improve¬ 
ment  approach,  Richard  McCabe  (of  the 
Systems  and  Software  Consortium)  and  I 
performed  an  analysis  of  potential  contra¬ 
dictions  between  CMMI-DEV  and  Lean 
software  development,  and  between 
CMMI-DEV  and  Agile  software  develop¬ 
ment.  We  rated  each  specific  practice  as 
Enabling,  Supportive,  Acceptable,  or  Unacceptable 
in  the  manner  in  which  they  supported  Lean 
and  Agile  software  development.  We  fuUy 
expected  to  find  a  few  to  dozens  of 
Unacceptables — but  found  none  [4]. 

This  result,  we  realized,  was  attributable 
to  the  wisdom  of  the  CMMI  framers,  who 
conceived  of  Required  (Goals),  Expected 
(Practices),  and  Informative  (everything  else) 
components.  Because  the  informative  com¬ 


ponents  can  be  adopted  according  to  the 
business  context  and  case,  and  because 
practices  can  be  modified  to  fit  the  business 
case,  the  CMMI-DEV  was  found  to  be  very 
supportive  of  both  Lean  and  Agile 
approaches. 

Lean  Thinking  is  about  a  fanatical  focus 
on  delivering  value  to  the  customer,  waste 
elimination,  setting  and  attaining  perfor¬ 
mance  goals,  cadence  and  synchronization, 
Agile  project  management,  and  fuUy  inte¬ 
grating  processes,  technologies,  and  knowl¬ 
edge  into  a  continuously  improving  frame¬ 
work  that  responds  quickly  to  customer 
demands.  In  Lean  organizations,  the 
processes  are  owned  by  the  doers  of  those 
processes,  and  they  are  charged  by  manage¬ 
ment  to  make  those  processes  perform  bet¬ 
ter  on  a  continuous,  sometimes  daily,  basis. 
Lean  software  development,  in  particular,  is 
well-defined  [1]. 

On  the  downside,  simultaneous  multiple 
Lean  efforts  (Kaizen  Events)  can  become 
uncoordinated  and  negatively  affect  one 
another.  Lean,  by  itself,  leaves  the  definition 
of  an  improvement  framework  or  infra¬ 
structure  to  the  organization. 

Lean  has  proven  fuUy  integrable  with 
the  CMMI,  ITIL,  and  Six  Sigma  efforts.  In 
most  cases.  Lean  is  treated  as  the  lead 
approach  since  it  offers  speed,  focus  on  cus¬ 
tomer  value,  and  responsiveness  to  cus¬ 
tomer  needs. 

Six  Sigma 

Six  Sigma  is  the  statistical  control  and  per¬ 
formance  prediction  capability  associated 
with  stable  processes.  A  process  that  is 
being  statistically  managed  enables  the  use 
of  project-level  leading  indicators  rather 
than  the  lagging  indicators  most  projects 
typically  use.  The  common  analogy  is  that 
managing  with  lagging  indicators  is  like 
driving  while  looking  into  the  rear-view 
mirror. 

A  Six  Sigma  process  is  simply  one  for 
which  the  specification  limits  are  six  stan¬ 
dard  deviations  from  the  central  tenden¬ 
cy — and  the  process  is  statistically  stable. 
Figure  2  (see  page  7)  illustrates  the  basics 
of  a  process  control  chart.  The  central 
tendency  is  normally  the  mean,  but,  in  cer¬ 
tain  situations,  tracking  the  mode  or  medi¬ 
an  may  prove  beneficial.  The  specification 
limits  are  set  to  the  desired  state  or  per¬ 
formance,  and  are  considered  the  voice  of 
the  customer.  The  natural  process  limits  are 
typically  set  to  ±  3  Sigma. 

After  processes  are  stabilized,  they  are 
made  to  perform  better  by  either  changing 
the  central  tendency  of  the  process  (typical¬ 
ly  the  mean),  and/or  reducing  the  variation 
due  to  common  causes.  Six  Sigma  tools 
include  regression  analysis,  tests  of  hypoth- 
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Software  Defense  Application 

For  more  than  a  decade,  the  author  has  used  CMMI  with  private  businesses  and 
government  defense  organizations  alike.  What  comes  from  these  experiences  is  an 
integrated  CMMI-based  framework  offering  improvements  for  defense  organiza¬ 
tions  in  focus,  speed,  and  cost  reductions.  The  real-world  results  speak  for  them¬ 
selves:  a  reduction  in  timelines  from  Maturity  Level  1  to  3  of  25-50  percent;  cost 
reductions  from  33-50  percent;  and  cost  avoidance  of  more  than  33  percent  for 
attainment  of  business  performance  goals  with  a  corresponding  Level  3  process 
capability. 


esis,  process  modeling  and  simulation, 
process  baselining,  process  control  charts, 
experimental  design,  and  optimization 
methods. 

On  the  downside,  process  control  is 
expensive  and  time-consuming.  It  depends 
on  process  execution  data,  so  longer 
improvement  cycle  times  (development, 
service,  or  acquisition)  can  be  problematic. 
Organizations  wishing  to  employ  Six  Sigma 
process  control  would  do  well  to  ensure  that 
the  predicted  return  on  investment  warrants 
the  investment. 

Six  Sigma  has  been  proven  to  be  fully 
integrable  with  CMMI  model  implementa¬ 
tions.  In  fact,  it  is  fair  to  say  that  Six  Sigma 
is  a  preferred  approach  for  implementation 
of  CMMI  high  maturity  (Capability  and 
Maturity  Levels  4  and  5).  Lean  has  the 
advantage  of  rapid  cycle  times,  which,  in 
addition  to  its  own  Lean  value,  offers  the 
benefit  of  rapid  data  collection  for  Six 
Sigma  studies. 

ITIL 

Version  3.0  of  the  ITIL  is  a  knowledge  base 
consisting  of  a  series  of  five  volumes: 
Service  Strategy,  Service  Design,  Service 
Transition,  Service  Operation,  and 
Continual  Service  Improvement.  Each  vol¬ 
ume,  in  addition  to  introductory  and  sum¬ 
mary  material,  is  composed  of  best  prac¬ 
tices  and  guidance,  including  risk  analyses. 

Certifications  associated  with  ITIL  are 
focused  on  individual  knowledge  development, 
from  the  Foundation  to  the  ITIL  Expert 
and  Master  levels.  For  organizations,  there  is 
one  recognized  and  one  emerging  option. 
Organizations  may  undergo  an  audit  against 
ISO  20000,  commonly  known  as  the  IT 
Service  Management  Standard.  This  stan¬ 
dard  is  somewhat  dated,  having  been  found¬ 
ed  largely  on  ITIL  2.0.  An  emerging  and 
promising  option  is  to  adopt  CMMI  for 
Services,  using  ITIL  3.0  best  practices  and 
guidance  as  informative  materials  in  the 
adoption  of  the  CMMI-SVC  model.  Several 
organizations  are  reportedly  targeting  on 
doing  SCAMPI  Class  A  benchmark 
appraisals  against  CMMI-SVC  with  inte¬ 
grated  ITIL  best  practices. 

On  the  downside,  ITIL  3.0  guidance  for 
the  improvement  infrastructure  is  relatively 
weak.  The  framework  itself  does  not  pro¬ 
vide  a  long-term  basis  for  consistent  and 
continuous  performance  improvement.  The 
existing  certification  standard  is  outdated,  and 
does  not  provide  a  structure  for  continu¬ 
ously  increasing  the  level  of  IT  service  man¬ 
agement. 

Several  examples  of  the  integration  of 
ITIL  3.0  into  an  implementation  of  CMMI- 
SVC  have  been  a  topic  of  some  interest  to 
the  CMMI  for  Services  Advisory  Group. 


The  marriage  appears  to  be  a  good  one, 
with  CMMI  providing  practices  for  the 
infirastructure  fior  improvement  and  the  appraisal 
method,  and  ITIL  providing  IT  service  best 
practices.  Several  SCAMPI  Class  B  and  C 
appraisals  have  reportedly  been  done 
against  the  CMMI-SVC  model,  with  ITIL 
best  practices.  Benchmark  (SCAMPI  Class 
A)  appraisals  are  being  planned. 

The  CMMI-Based  Integrated 
Framework 

As  mentioned  throughout  this  article,  each 
CMMI  model  provides  a  high-value  frame¬ 
work  for  integration  of  other  improvement 
approaches.  Figure  3  reveals  the  most 
salient  attributes  of  each  improvement 
approach,  and  how  these  attributes  overlap 
and  support  one  another.  For  example. 
Lean  Thinking  provides  a  sharp  degree  of 
focus  on  customer  value,  and  provides 
mechanisms  for  rapid  improvement. 

Figure  3  also  indicates  that  CMMI  is, 
practically  speaking,  collapsed  in  its  applica¬ 
tion  by  applying  Lean  Thinking  to  its  imple¬ 
mentation.  Six  Sigma  is  basically  orthogonal 
(independent)  of  both  Lean  and  CMMI, 
and  works  well  with  both  approaches.  As 
described  previously,  ITIL  is  treated  as  an 
extension  of  the  infiormative  components  in 
CMMI-SVC. 

Earlier,  I  discussed  the  three  driving 
principles  of  performance  improvement; 
Figure  4  illustrates  that  these  principles 
form  the  focus  of  how  the  improvement 
approaches  are  integrated  into  the  CMMI 
framework.  In  all  cases  in  which  we  have 
enjoyed  measurable  success,  it  provides  the 
basic  framework  for  improvement. 

As  well.  Figure  4  depicts  the  mind  map  of 
working  through  the  integration  of  these 
improvement  approaches  for  a  particular 
environment  or  business  domain,  and  re¬ 
minds  us  of  the  importance  of  fimt principles. 

This  integrated  framework  for  perfor¬ 
mance  excellence  has  become  the  de  facto 
approach  for  improvement  in  several  of  our 
segments,  and,  we  believe,  will  offer  increas¬ 
ing  value  and  responsiveness  as  we  continue 
learning  how  to  tune  this  framework.  But 
we  can  already  appreciate  that  our  efforts, 
and  those  of  many  others,  have  helped  to 


get  the  most  out  of  CMMI.^ 
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Process  Improvement  for  All: 

What  to  Expect  from  CMMI  Version  1.3 

Mike  Phillips  and  Sandy  Shrum 
Software  Engineering  Institute 

The  next  rekase  of  the  CMMI  Product  Suite — an  approach  that  provides  organisations  with  the  essential  ekments  of  effec¬ 
tive  processes  that  ultimately  improve  their  performance — is  expected  in  November  2010.  This  Version  1.3  (V1.3)  release 
includes  improvements  to  CMMI for  Development  (CMMI-DEV),  CMMI for  Acquisition  (CMMI-ACQ),  and  CMMI 
for  Services  (CMMI-SVC)  models  all  during  the  same  development  cycle.  This  cycle  also  includes  improvements  to  the 
appraisal  method  (SCAMPI)  and  CMMI-related  training.  The  improvements  planned  for  CMMI  models  do  not  require 
major  changes  or  retraining  for  those  currently  using  CMMI. 


Since  2000,  the  CMMI  Product  Suite — 
through  three  separate  models — has 
given  organizations  a  framework  for 
improving  their  processes  (see  Figure  1). 
First  was  the  CMMI-DEV  model  (created 
in  2000  and  updated  in  2002  and  2006), 
which  helps  product  and  service  develop¬ 
ment  organizations  integrate  their  software 
and  systems  engineering  while  improving 
their  processes  and  performance.  The 
CMMI-ACQ  model  was  then  released  in 
2007  to  help  organizations  that  outsource, 
acquire,  purchase,  or  otherwise  acquire 
products  and  services  for  their  customers. 
The  most  recent  model,  CMMI-SVC,  was 
released  in  2009.  It  helps  service  organiza¬ 
tions  to  develop  quality  service  processes 
that  enable  improved  performance,  cus¬ 
tomer  satisfaction,  and  profitability. 

During  all  of  this  development  work  on 
a  product  Kne,  these  models  were  used  by 
various  organizations,  and  some  organiza¬ 
tions  even  used  more  than  one  CMMI 
model.  All  three  models  follow  the  same 
structure,  philosophy,  and  general  approach. 
Furthermore,  there  are  details  that  are  com¬ 
mon  across  all  three  models. 

Even  though  these  models  were 
released  in  different  years.  Version  1.2 
(VI. 2)  was  the  last  release  of  all  three 
CMMI  models.  Two  major  themes  drove 
the  changes  that  comprised  VI. 2: 

I.  Refining  the  CMMI  model  architecture 
to  create  CMMI  constellations  that 
served  areas  of  interest  (i.e..  Develop¬ 
ment,  Acquisition,  Services).  This 
change  resulted  in  the  creation  of  the 
CMMI-ACQ  and  CMMI-SVC  models. 

2.  Improving  the  integrity  of  SCAMPI 
appraisals  that  use  CMMI  models  as  the 
reference  model  to  measure  process 
improvement  achievement.  SCAMPI 
appraisals  are  events  that  follow  a  stan¬ 
dard  method  for  evaluating  how  well  an 
organization’s  processes  conform  to  the 
practices  in  a  CMMI  model.  When  you 
use  a  CMMI  model  and  conduct  a 
SCAMPI  appraisal,  you  receive  appraisal 


results  that  reflect  your  organization’s 
maturity  and  capability.  Beginner  organi¬ 
zations  new  to  CMMI  are  typically  con¬ 
sidered  low  maturity,  while  those  that 
have  achieved  exemplary  appraisal 
results  are  considered  high  maturity. 
Before  the  idea  for  a  VI. 3  release  was 
settled  on,  the  development  team  created 
and  reviewed  model  updates  that  could  be 
released  as  CMMI  Version  1. 2a  (VI. 2a). 
This  version,  considered  a  minor  update, 
was  to  include  updates  made  only  to  infor¬ 
mative  material'. 

The  planned  model  updates  for  VI. 2a 
were  primarily  to  clarify  high  maturity  prac¬ 
tices.  These  updates  were  reviewed  by  a 
group  of  CMMI  High  Maturity  Lead 
Appraisers  and  the  CMMI  Steering  Group 
(the  executive  committee  that  guides  all 
CMMI  work)  at  a  workshop  in  late 
September  2008.  As  a  result  of  the  review, 
the  Steering  Group  determined  that  making 
changes  to  the  normative  material  to  mod¬ 
ernize  the  practices  for  Maturity  Levels  4 
and  5  was  a  better  choice  than  only  clarify¬ 
ing  the  practices  by  updating  informative 
material.  So  rather  than  releasing  CMMI- 
DEV  VI. 2a,  the  development  team  is 
including  these  and  other  model  updates  in 
the  planned  release  of  CMMI  VI. 3  for  all 
three  CMMI  models  (CMMI-DEV,  CMMI- 
ACQ,  and  CMMI-SVC). 

The  Development  ofVI.3 

The  CMMI  VI. 3  project  was  initiated  in 
January  2009  when  the  plan  to  update  the 
CMMI  Product  Suite  was  announced.  The 
plan  included  two  months  for  users  to  pro¬ 
vide  final  change  requests  before  the  devel¬ 
opment  team  would  begin  reviewing  and 
analyzing  of  the  submitted  requests. 

During  March  through  June  of  2009, 
the  development  team  reviewed  more 
than  1,150  change  requests  submitted  for 
the  three  CMMI  models  and  850  change 
requests  for  the  SCAMPI  appraisal 
method.  Teams  were  formed  to  initiate 
the  development  of  VI. 3. 


From  March  until  June,  the  CMMI 
Steering  Group  provided  criteria  to  guide 
the  range  of  acceptable  changes  to  the 
CMMI  Product  Suite.  The  “CMMI  Version 
1.3  -  Plans  for  the  Next  Version”  [I]  was 
published  by  the  SEI  in  August  2009.  It 
stated  that  it  will  focus  on  (but  not  be  limit¬ 
ed  to): 

1 .  High  maturity. 

2.  More  effective  GPs. 

3.  Appraisal  efficiency. 

4.  Commonality  across  the  constellations. 
It  also  required  that  any  changes  to  the 

CMMI  Product  Suite  (i.e.,  model(s),  training 
materials,  and  appraisal  method)  must  meet 
the  following  primary  criteria,  which  will 
likely  do  the  following  (also  from  [I]): 

1 .  Correct  identified  model,  training  mate¬ 
rial,  or  appraisal  method  defects  or  pro¬ 
vide  enhancements. 

2.  Incorporate  amplifications  and  clarifica¬ 
tions  as  needed. 

3.  Accommodate  potential  additions  to 
model  coverage  (e.g,  safety,  security,  and 
life  cycle)  only  by  specific  direction  of 
the  CMMI  Steering  Group. 

4.  Decrease  overall  model  size  in  VI. 3  if 
possible;  increases,  if  any,  must  not  be 
greater  than  absolutely  necessary. 

5.  Model  and  method  changes  should 
avoid  adversely  impacting  the  legacy 
investment  of  adopting  companies  and 
organizations. 

6.  Changes  to  model  architecture  will  only 
be  incorporated  with  specific  CMMI 
Steering  Group  authorization. 

7.  Changes  can  only  be  initiated  by  Change 
Requests  or  by  the  CMMI  Steering 
Group. 

8.  Editorial  changes  to  training  may  be 
released  in  advance  of  VI. 3. 

9.  Changes  must  not  require  retraining  the 
nearly  100,000  (as  of  Dec.  2008)  per¬ 
sonnel  already  trained  in  CMMI. 
Upgrade  training  may  be  needed,  espe¬ 
cially  for  instructors,  lead  appraisers,  and 
appraisal  team  members. 

Each  of  the  recent  CMMI  releases  has 
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been  guided  by  criteria  for  acceptable 
change  provided  by  the  CMMI  Steering 
Group.  These  criteria  are  typically  similar  to 
the  criteria  in  [1];  however,  each  set  of  crite¬ 
ria  also  has  some  aspects  that  characteri2e 
the  version  being  released. 

The  correction  of  defects  is  an  obvi¬ 
ous  reason  for  change.  The  SEI  has  iden¬ 
tified  corrections  with  errata  sheets  pub¬ 
lished  on  their  Web  site  between  formal 
releases.  These  corrections  are  then  incor¬ 
porated  into  the  next  version.  Besides  out¬ 
right  corrections,  submitters  of  change 
requests  submit  what  they  think  are 
improvements  to  the  model.  These 
improvements  are  often  clarifications  of 
existing  model  material.  The  second  crite¬ 
rion  encourages  clarifications  that  may  be 
needed  to  fuUy  understand  the  intent  of 
model  goals  and  practices. 

Excessive  model  growth  is  a  significant 
concern,  and  therefore  criteria  3  and  4  seek 
to  limit  additions  to  this  release.  These  cri¬ 
teria  couple  nicely  with  criterion  5,  which 
reminds  the  team  to  protect  the  legacy 
investment  of  the  thousands  of  organiza¬ 
tions  who  are  using  the  CMMI  Product 
Suite  already.  Criterion  9  adds  a  further  con¬ 
straint  so  that  no  one  will  have  to  start  over 
with  the  Introduction  to  CMMI  course  sim¬ 
ply  because  of  the  release  of  VI. 3. 

The  Major  Elements  of VI. 3 

Many  improvements  will  be  incorporated 
into  the  CMMI  Product  Suite  for  VI. 3. 
Some  of  the  more  significant  improve¬ 
ments  are  described  here. 

High  Maturity  Clarifications 

As  already  mentioned,  when  you  conduct  a 
SCAMPI  appraisal,  you  receive  appraisal 
results  that  reflect  your  organization’s  matu¬ 
rity.  Beginner  organizations  that  are  new  to 
CMMI  are  typically  considered  low  maturi¬ 
ty  while  those  that  have  achieved  exemplary 
appraisal  results  are  considered  high  maturi¬ 
ty.  A  focus  of  current  model  development  is 
on  clarifying  the  practices  associated  with 
high  maturity  for  organizations  using  the 
staged  approach — and  high  capability  in 
process  areas  (PAs)  for  organizations  using 
the  continuous  approach^. 

A  High  Maturity  Team  was  formed. 
This  team’s  members  have  been  focusing  on 
making  changes  that  improve  the  clarity  of 
what  high  maturity  is  and  providing  the 
guidance  needed  to  achieve  it.  A  team  leader 
was  chosen  from  industry  project  partici¬ 
pants  to  ensure  that  the  improvements 
made  are  representative  of  current  best 
practices  in  the  community. 

The  High  Maturity  Team  recognized 
that  high  maturity  practices  are  currently 
unclear,  leading  to  a  variety  of  interpreta¬ 


tions  by  users.  As  they  work  on  VI. 3,  the 
team’s  main  objective  is  to  ensure  that  aU 
CMMI  users  have  a  common  understand¬ 
ing  of  high  maturity  practices  in  aU  three 
models. 

Thus  far,  the  team  intends  to  clarify  the 
following: 

•  The  role  of  informative  material  in  high 
maturity  appraisals. 

•  The  meaning  and  use  of  process  mod¬ 
els  and  process  modeling. 

•  How  business  objectives  are  related  to 
and  lead  to  high  maturity. 

•  What  common  causes  are  and  how  they 
are  expected  to  be  used. 

•  What  high  maturity  expectations  are  on 
individual  PA  performance. 

•  The  selection,  definition,  and  level  of 
instantiation  of  subprocesses. 

The  high  maturity  changes  to  the  infor¬ 
mative  material,  produced  in  the  VI. 2a 
effort  mentioned  earlier,  are  only  a  part  of 
the  full  array  of  change  requests  now  being 
reviewed  by  this  team  in  its  VI. 3  model 
development  effort.  Also  planned  are 
changes  to  the  structure  of  high  maturity  in 
the  model,  which  includes  changes  that 
strengthen  the  alignment  between  Maturity 
Level  4  and  5  practices. 

This  team’s  work  focuses  on  the  high 
maturity  PAs:  Organizational  Process  Per¬ 
formance,  Quantitative  Project  Manage¬ 
ment,  Causal  Analysis  and  Resolution,  and 
Organizational  Innovation  and  Deploy¬ 
ment. 

Constellation  Commonality 

As  the  development  team  built  on  the  con¬ 


tent  of  the  CMMI-DEV  model  to  create 
the  CAIMI-ACQ  and  CAIMI-SVC  models, 
it  modernized  some  of  the  material  in  the 
16  core  PAs  that  are  common  to  aU  three 
constellations.  This  modernization  meant 
that  the  development  team  knew  it  had  to 
eventually  revisit  CMMI-DEV  and  modern¬ 
ize  it  as  well.  Even  though  some  differences 
between  the  constellations  is  intentional  and 
makes  sense,  some  of  the  differences  can  be 
eliminated  to  make  the  three  constellations 
more  consistent  and  easier  to  use  together. 

Here  are  some  examples  of  the  changes 
being  made  to  improve  commonality  across 
the  three  CMMI  models: 

•  Core  PAs.  These  are  PAs  that  appear 
in  all  CMMI  models.  In  VI. 3,  these 
core  PAs  can  have  different  expected 
and  informative  material.  For  example. 
Project  Planning  can  have  a  specific 
practice  in  the  Service  constellation 
that  is  absent  in  the  version  of  the  PA 
in  the  Development  constellation. 
Likewise,  a  few  PAs  are  shared  and 
appear  in  more  than  one  (but  not  aU) 
models.  Shared  PAs  also  can  have  dif¬ 
ferent  expected  and  informative  mate¬ 
rial.  However,  work  has  been  done  to 
ensure  that  core  PAs  are  as  common  as 
it  makes  sense  for  them  to  be.  If  mate¬ 
rial  can  work  well  in  all  three  models,  it 
is  made  consistent.  If  not,  the  material 
remains  different. 

•  Teaming.  There  are  two  different 
approaches  to  integrated  teaming  in 
CMMI  models:  in  CMMI-DEV,  team¬ 
ing  is  covered  in  two  goals  which  are 
treated  as  optional,  or  additions;  in 
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CAIMI-ACQ  and  CAIMI-SVC,  teaming 
is  covered  in  two  specific  practices  in 
two  process  areas  (Organizational 
Process  Definition  and  Integrated 
Project  Management).  These  practices 
are  expected  model  components  and 
are  not  optional. 

In  CMMI  VI. 3,  the  development  team 
determined  that  the  best  approach  to  use 
in  aU  three  CMMI  constellations  is  the  one 
used  in  the  CMMI-ACQ  and  CMMI-SVC 
models.  This  work  to  ensure  commonality 
of  the  approach  to  teaming  has  gained 
importance  since  the  Team  Software 
Process  has  demonstrated  the  perfor¬ 
mance  potential  of  high  performing 
teams. 

•  PA  Categories.  There  are  six  PA  cate¬ 
gories  for  VI. 3:  Process  Management, 
Project  Management,  Support,  Engi¬ 
neering,  Acquisition,  and  Service 
Establishment  and  Delivery.  AU  PAs 
that  are  core  must  have  the  same  PA 
category  in  all  three  models  and  this 
PA  category  must  be  one  of  the  fol¬ 
lowing:  Process  Management,  Project 
Management,  or  Support.  PAs  that  are 
not  core  must  be  assigned  to  one  of 
the  following  PA  categories:  Engi¬ 
neering,  Acquisition,  or  Service  Estab- 
Ushment  and  Delivery.  As  a  result. 
Requirement  Management  will  be 
assigned  to  the  Project  Management 
PA  category  in  aU  V1.3  CMMI  models. 

•  Generic  goals  and  practices.  In  VI. 2 
models,  generic  goals,  generic  prac¬ 
tices  (GPs),  and  GP  elaborations  are 
presented  differently  across  models. 
The  CMMI-DEV  model  presents  a 
portion  of  these  elements  in  Part  One 
and  others  are  included  at  the  end  of 
each  PA  in  Part  Two.  In  CMMI-ACQ 
and  CMMI-SVC,  these  elements 
appear  in  a  single  section  in  Part  Two 
before  the  PAs.  In  VI. 3,  these  generic 
elements  will  all  appear  in  all  three 
models  in  one  central  location  as  the 
first  section  of  Part  Two. 

•  Glossary.  The  glossaries  in  all  three 
models  have  become  inconsistent  sim¬ 
ply  because  of  the  gaps  between  pub- 
Ucation  dates  of  the  models.  In  VI. 3 
models,  the  glossaries  will  be  exactly 
the  same,  even  though  some  terms 
defined  may  not  appear  in  one  or  more 
of  the  models.  The  format  of  the  glos¬ 
sary  wiU  also  be  modified  to  differenti¬ 
ate  the  definition  from  the  notes. 

Modernized  Practices 

Improvements  to  the  practices  in  multiple 
process  areas  wiU  be  updated  to  ensure 
they  are  modern  and  reflect  the  best  prac¬ 
tices  available. 


•  Agile.  Material  will  be  added  to  the 
model  to  help  those  in  Agile  environ¬ 
ments  to  correctly  interpret  practices 
that  may  not  seem  appUcable. 

•  Architecture-Related  Development. 
Material  will  be  updated  and  added  to 
include  the  consideration  of  both  non¬ 
functional  and  functional  require¬ 
ments  during  product  development. 

•  Supplier  Agreement  Management. 
The  scope  of  suppUer  agreement  man¬ 
agement  will  be  clarified,  particularly 
in  regards  to  COTS,  internal  sourcing, 
and  customer  property. 

•  Organizational  Training.  Organi¬ 
zational  training  practices  will  be 
updated  so  that  they  apply  to  more 
than  classroom  instruction. 

Translations 

CMMI  models  are  now  available  in 
French,  German,  Japanese,  Spanish,  and 
traditional  Chinese.  By  the  time  this  article 
is  published,  a  version  in  Portuguese  will 
also  be  available.  The  teams  that  created 
these  translations  have  requested  that  the 
models’  ease  of  translation  be  improved. 
A  simple  example  of  a  change  that  can  be 
made  to  the  model  to  ease  its  translation  is 
eliminating  the  use  of  the  word  stovepipe. 
This  word  is  one  of  many  that  are  difficult 
to  interpret  into  different  languages 
appropriately  because  its  literal  meaning  is 
different  from  how  it  is  used  in  CMMI 
models. 

Expanded  Coverage 

A  number  of  change  requests  have  sug¬ 
gested  further  expansion  of  CMMI  mod¬ 
els  in  new  areas.  The  CMMI  Steering 
Group  and  the  development  team  do  not 
see  this  release  as  being  suitable  for  major 
expansions  like  the  addition  of  the  two 
recent  constellations,  but  the  team  will 
likely  add  updated  information  on  archi¬ 
tecture,  software  assurance,  Agile,  and 
Lean  Six  Sigma.  The  development  team 
has  also  been  encouraged  to  add  more 
emphasis  on  customer  satisfaction.  These 
types  of  expansions  modernize  model 
coverage  without  adding  new  PAs. 

Multi-Constellation  Coverage 

Many  students  who  take  either  of  the  one- 
day  supplement  courses  that  cover  acqui¬ 
sition  or  service  delivery  have  commented 
that  many  organizations  span  more  than 
one  area  of  interest.  One  theme  for  the 
VI. 3  release  is  to  enable  as  much  sharing 
of  best  practices  across  the  constellations 
as  possible.  Once  some  effective  pilots  are 
conducted,  the  development  team  plans  to 
improve  the  SCAMPI  Method  Definition 
Document  to  facilitate  appraisals  that  use 


PAs  from  multiple  constellations. 

Appraisal  Efficiency 

The  SCAMPI  appraisal  method  was  based 
(in  part)  on  the  CMM-Based  Appraisal  for 
Internal  Process  Improvement  (CBA-IPI) 
assessment  method  used  with  the  Software 
CMM,  a  predecessor  of  CMMI.  The 
SCAMPI  appraisal  method  moved  from  the 
discovery  focus  of  CBA-IPI  to  a  verifica¬ 
tion  focus.  This  change  was  designed  to 
save  significant  appraisal  time.  The  Practice 
Implementation  Indicator  Documents 
(PIIDs)  were  introduced  to  reduce  on-site 
appraisal  time.  These  documents  list  work 
products  that  the  appraisal  team  can  look 
for  as  evidence  that  a  practice  was  imple¬ 
mented.  However,  the  development  team  is 
investigating  whether  organizations  are 
spending  excessive  time  preparing  PIIDs.  If 
they  are,  the  development  team  will  exam¬ 
ine  ways  of  upholding  appraisal  confidence 
without  driving  up  preparation  expenses. 
The  SCAMPI  upgrade  team  is  looking  for 
innovative  ways  to  achieve  this  goal. 

Other  improvements  to  the  SCAMPI 
Method  Definition  Document  that  wiU  like¬ 
ly  be  included  in  VI. 3  include: 

•  Providing  SCAMPI  support  for  aU  three 
CMMI  models  by  removing  problemat¬ 
ic  terminology,  addressing  appraisal 
scoping  considerations,  and  identifying 
appropriate  prerequisites  for  appraisal 
team  members. 

•  Correcting  all  errors  identified  during 
the  use  of  SCAMPI  VI. 2,  including 
common  pitfalls  encountered  by  users 
and  problems  frequently  encountered  in 
reviews  of  appraisals  by  the  SEI. 

•  Clarifying  the  meaning  of  focus  and 
non-focus  projects  as  well  as  direct  and 
indirect  artifacts. 

•  Clarifying  guidelines  for  scoping 
appraisal  in  a  wide  range  of  organiza¬ 
tion  types  and  sizes. 

•  Providing  guidelines  to  ensure  consis¬ 
tent  handling  of  GPs. 

•  Resolving  issues  related  to  characteriza¬ 
tion  rules  and  rating  rules. 

Model  Sizing 

To  meet  the  fourth  criterion  that  limits  the 
overall  size  of  CMMI  models,  the  develop¬ 
ment  team  looks  for  ways  to  balance  model 
additions  with  deletions.  Feedback  resulting 
from  an  effort  collecting  input  from  multi¬ 
ple  lead  appraisers  called  ATLAS’ — short 
for  “Ask  The  Lead  Appraisers,”  facilitated 
by  Pat  O’Toole — was  received.  This  group 
submitted  change  requests  that  identified 
lower  value  practices  that  might  be  removed 
to  add  others  now  viewed  as  more  impor¬ 
tant.  As  mentioned  earlier,  additional  PAs 
are  not  encouraged  for  VI. 3. 
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Figure  2:  CMMI  V1.3  Model  Schedule 


Upgrade  to  VI. 3 

The  CMMI  Steering  Group  has  approved 
a  significant  period  of  overlap  between 
the  release  of  CMMI  VI. 3  and  the  retire¬ 
ment  of  CMMI  VI. 2.  The  development 
team  is  also  investigating  innovative  ways 
of  providing  information  about  CMMI 
improvements  to  users  in  draft  form. 
However,  no  one  is  encouraged  to  delay 
their  process  improvement  programs 
just  to  wait  for  the  release  of  VI. 3. 

VI. 3  Training 

Training  will  be  developed  that  will  pro¬ 
vide  an  easy  upgrade  from  VI. 2  to  VI. 3 
for  aU  three  models.  This  training  will  be 
made  available  online.  The  Introduction 
to  CMMI  course  will  be  updated  as  will 
the  three-day  Introduction  to  CMMI- 
SVC.  The  current  supplement  courses 
for  CMMI-ACQ  and  CMMI-SVC  wiU  be 
retained  and  updated  appropriately. 

Development  Schedule  for  VI. 3 

Given  the  number  of  change  requests 
and  the  span  of  contentious  issues  to  be 
resolved  by  the  teams,  the  development 
team  has  been  concerned  about  declar¬ 
ing  a  schedule  for  VI. 3.  The  current  esti¬ 
mate  is  to  release  the  three  constellations 
by  November  1,  2010,  but  this  date 
assumes  few  changes  are  needed  after 
analyzing  feedback  on  the  drafts.  Figure 
2  provides  a  high-level  view  of  the  VI. 3 
schedule  for  the  models,  and  Figure  3 
provides  a  similar  view  of  the  schedule 
for  the  SCAMPI  improvements. 

Figure  2  shows  that  the  model  devel¬ 
opment  project  started  in  January  2009 
and  will  conclude  in  November  2010. 
Preparation  activities  included  planning, 
forming  teams,  and  defining  processes 
and  occurred  from  January  to  May  2009. 
Following  that,  from  June  to  October 

2009,  change  packages  (CPs)  were  creat¬ 
ed,  reviewed,  and  approved.  CPs  are 
descriptions  of  planned  change  based  on 
change  requests  received;  they  are 
reviewed  and  approved  by  the  develop¬ 
ment  team  and  then  by  the  CMMI 
Configuration  Control  Board  (CCB), 
which  is  responsible  for  controlling 
change  to  the  CMMI  Product  Suite. 

Once  these  are  approved,  actual 
changes  to  model  components  are  pro¬ 
posed  in  redlines.  These  redlines  are 
scheduled  to  be  created,  reviewed,  and 
approved  from  August  2009  to  April 

2010.  These  redlines  are  also  reviewed 
and  approved  by  the  development  team 
and  CCB. 

Piloting  is  also  scheduled  to  enable 
organizations  willing  to  pilot  improve¬ 
ments  before  release  and  provide  feed¬ 


back  on  their  utility.  There  are  three  pilot 
drafts  currently  planned:  a  November 

2009  draft  included  changes  that 
improve  the  consistency  of  all  three 
models  with  one  another,  a  January  2010 
draft  will  include  many  of  the  model 
improvements  (in  particular  all  of  the 
high  maturity  improvements),  and  a  June 

2010  draft  wiU  include  aU  model  changes 
minus  the  poUshing  provided  by  quaUty 
assurance  (QA).  Updates  based  on  feed¬ 
back  from  piloting  wiU  be  made  from 
May  to  July  of  2010.  Finally,  QA  wiU 
begin  in  July  to  prepare  the  models  for 
November  2010  release. 

By  the  time  this  article  is  pubUshed, 
much  of  the  development  of  model 


improvements  should  be  determined  and 
piloting  wUl  have  begun.  If  your  organi¬ 
zation  is  interested  in  participating  in 
piloting  by  reporting  on  your  use  of 
draft  versions  of  a  CMMI  VI. 3  draft 
model  (CMMI-DEV,  CMMI-ACQ, 
CMMI-SVC),  contact  SEI  customer  rela¬ 
tions  at  <customer-relations@sei. 
cmu.edu>.  The  development  team  wiU 
send  you  details  of  how  you  can  receive 
drafts  and  how  to  provide  structured 
feedback. 

Figure  3  shows  the  CMMI  VI. 3 
SCAMPI  Method  Definition  Document 
development  project  schedule,  which 
parallels  the  previously  described  model 
development  effort. 


Figure  3:  GMMI  V  1.3  SCAMPI  Upgrade  Schedule 
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CMMI:  Getting  a  Handle  on  Process 


Software  Defense  Application 

Since  many  DoD  and  defense  contractor  organizations  within  the  software  commu¬ 
nity  are  currendy  utilizing  CMMI  Product  Suite  VI. 2,  this  article  is  an  extremely  valu¬ 
able  primer  for  the  approaching  upgrade  to  VI. 3.  The  value  of  this  article  is  to  alert 
the  software  defense  community  to  the  specific  improvements  to  the  three  models 
(CMMI-DEV,  CMMI-ACQ,  and  CMMI-SVC),  the  SCAMPI  appraisal,  and  CMMI 
training  methods.  Time  and  money  may  be  saved  by  knowing  what’s  coming,  and 
perhaps  by  participating  in  the  SETs  pilot  program  of  the  draft  models  (discussed  in 
the  Development  Schedule  for  VI. 3  section). 

Summary  library/ abstracts/news-at-sei/ cmmiin 

The  improvements  included  in  CMMI  VI. 3  focus200904.cfm>. 

include  high  maturity  improvements  and 
clarifications,  improved  appraisal  efficiency.  Notes 

and  models  that  have  consistent  architec-  1.  The  components  that  comprise  CMMI 
tures  and  shared  content  At  this  point  in  models  are  grouped  into  three  cate- 

the  development  life  cycle,  the  improve-  gories:  required,  expected,  and  informa- 

ments  have  not  required  major  changes  to  tive.  Unlike  the  required  and  expected 

the  existing  VI. 2  product  suite.  Therefore,  model  components,  the  informative 

CMMI  users  should  continue  with  their  model  material  is  not  considered  nor- 

process  improvement  programs  without  mative. 

regard  to  the  release  date  for  VI. 3.^  2.  These  two  approaches  are  variations  in 

how  appraisals  are  conducted.  However, 
Reference  the  idea  of  high  maturity  is  essentially 

1.  Phillips,  Mike.  “CMMI  Version  1.3 —  the  same  in  both. 

Plans  for  the  Next  Version.”  Nem  at  3.  Learn  more  about  ATLAS  at  <www. 

SEl.  1  Aug.  2009  <www.sei.cmu.edu/  pactcmmi.com/pages/atias>. 
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Scope  Management:  1 2  Steps  for  ICT  Program  Recovery® 

Carol  Dekkers  Pekka  ForseHus 

Qualily  Plus  Technologies,  Inc.  4 SUM  Partners,  Inc. 

The  information  and  communications  technology  (ICT)  world  is  “addicted”  to  dysfunctional  behavior  and  the  problem  is 
spreading  globally.  The  sad  truth  is  that  the  parties  in  the  ICT  relationship  (the  customer  and  the  supplier)  are  largely  co¬ 
dependent  on  a  pattern  of  dysfunction  characterised  by  ineffective  communication,  fixed  price  contracts  with  changing  require¬ 
ments,  and  eroding  trust.  This  article  focuses  specifically  on  the  northerns  COPE™  E-step  process  for  ICT  program  recovery. 


The  ICT  world’s  dependence  on  dys¬ 
functional  behavior — specifically 

ineffective  communication,  fixed  price 
contracts  with  changing  requirements, 
and  eroding  trust — is  devastating. 
Billions  of  dollars  are  being  spent  on 
rework  and  enhancements  to  (defective) 
software  [1].  The  latest  CHAOS  report 
says  that  a  staggering  two-thirds  of  ICT 
projects  are  deemed  failures  [2],  with 
schedule  extensions,  cost  overruns,  and 
poor  quality  software  more  the  norm 
than  the  exception  to  the  rule.  One  can 
find  daily  news  stories  where  ICT  pro¬ 
jects  are  reportedly  over  budget  by  sever¬ 
al  hundred  percent,  overdue  by  years,  or 
cancelled  after  millions  of  dollars  have 
already  been  spent. 

Almost  a  decade  ago,  software  suppli¬ 
ers  recognized  their  role  in  this  problem 
and  started  investing  in  process  improve¬ 
ment.  Similarly,  customers  focused  on 
improving  their  technical  knowledge  in 
the  hope  that  they  could  better  direct 
suppliers  to  implement  the  right  solution. 
Professional  project  management  and 
software  process  improvement  initiatives 
helped  streamline  the  supplier  side  of 
ICT  programs  and  projects.  However,  the 
issues  at  play  are  systemic  and  involve 
customers  and  suppliers. 

Questions  of  how  to  improve  this 
state  of  affairs — in  an  industry  of 
advanced  technology,  bright  project  man- 


Figure  1:  Scope  Management  of  northern- 
SCOPE  and  the  PMBOK  Guide  Knowledge 
Areas  [1 1] 
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agers,  and  leading-edge  maturity  mod¬ 
els — ^led  both  Australia  and  Finland  to 
individually  investigate  further,  with 
results  worthy  of  attention.  Their  formal¬ 
ized  scope  management  approaches — 
southernSCOPE  [3]  and  northern- 
SCOPE  [4] — have,  within  the  first  few 
years,  reversed  the  trend  of  failed  pro¬ 
jects,  posted  increased  ICT  program  suc¬ 
cess,  and  improved  customer/ supplier 
relationships.  Both  approaches  are  based 
on  project  management  best  practices 

**Scope  management 
is  central  to  software 
development  and  must 
be  integrated 
especially  in  regards  to 
time,  cost,  quality, 
and  risk  management/* 


combined  with  customer-centric  scope 
management,  and  illustrate  the  important 
role  of  scope  management  in  building 
more  resilient  software  that  can  stand  the 
test  of  time'. 

Both  initiatives  examine  and  advance 
ICT  programs  through  steps  that  initial¬ 
ize,  scope,  split  into  manageable  sub-pro¬ 
jects  (as  necessary),  quantify  size,  cost  (on 
the  basis  of  currency  per  unit  size),  and 
manage  and  deliver  through  professional 
ICT  scope  management.  The  results  of 
both  approaches  are  profound:  Success 
rates  on  ICT  projects  have  skyrocketed 
and  cost  overruns  have  plummeted  to 
levels  unprecedented  in  the  ICT  industry. 
In  2005,  the  International  Software 
Benchmarking  Standards  Group  (ISBSG) 
proclaimed: 

©  Carol  Dekkers  and  Pekka  Forselius.  All  rights  reserved. 

™  northernSCOPE  is  registered  in  the  U.S.  Patent  and 

Trademark  Office  to  FiSMA. 


...  the  cost  overruns  for  projects 
using  the  southernSCOPE  method 
were  found  to  be  less  than  10  per¬ 
cent  whereas  the  industry  norm 
was  84  percent.  [9] 

This  article  focuses  on  the  concepts  of 
the  more  recent  northernSCOPE,  and  the 
differences  between  it  and  southern¬ 
SCOPE  will  be  noted  as  appropriate. 
Additionally,  the  new  job  role  of  a  certi¬ 
fied  Scope  Manager  (CSM),  as  established 
by  the  European  Certification  &  Quali¬ 
fication  Association,  is  discussed.  Scope 
management  is  not  rocket  science;  how¬ 
ever,  managing  scope  is  not  a  natural 
byproduct  of  project  management. 

Why  Scope  Management? 

Introduced  in  “A  Guide  to  the  Project 
Management  Body  of  Knowledge” 
(PMBOK  Guide)  as  a  knowledge  area, 
scope  management  can  be  more  impor¬ 
tant  to  project  success  than  any  of  the 
other  individual  knowledge  areas.  As  a 
case  in  point,  60-99  percent  of  all  defects 
latent  in  production  software  can  be 
attributed  to  the  requirements  phase  [10]. 
While  project  scope  management  will  not 
guarantee  perfect  requirements,  the  simple 
act  of  identifying  scope  delineates  what  is 
within  the  requirements  and  what  is  not. 

Scope  management  effectively 
addresses  five  out  of  six  of  the  most 
common  reasons  cited  for  ICT  project 
cost  overruns  and  uncontrolled  project 
growth  [8]: 

1 .  Lack  of  user  input. 

2.  Incomplete  requirements. 

3.  Changing  requirements. 

4.  Technology  incompetence. 

5.  Unrealistic  expectations. 

Scope  management  is  critical  to  suc¬ 
cessful  ICT  project  completion. 
northernSCOPE  places  scope  manage¬ 
ment  dead  center  in  the  overall  PMBOK 
Guide  knowledge  areas  because  it 
involves  and  interfaces  with  all  eight  areas 
(as  depicted  in  Figure  1) . 

The  PMBOK  Guide  definition  of 
project  scope  management  is  “...  the 
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Figure  2:  Mapping  northernSCOPE  Processes  against  PMBOK  Guide  Process  Groups 


processes  required  to  ensure  that  the  pro¬ 
ject  includes  all  the  work  required,  and 
only  the  work  required,  to  complete  the 
project  successfully”  [12]^.  Scope  manage¬ 
ment  is  central  to  software  development 
and  must  be  integrated  especially  in 
regards  to  time,  cost,  quality,  and  risk 
management.  There  are  no  scope  changes 
without  possible  consequences  to  sched¬ 
ule,  budget,  and  quality,  or  increases  to  the 
risk  level  of  the  project.  This  is  true  vice- 
versa  as  well:  If  the  schedule  or  budget 
must  be  tightened,  it  requires  changing 
the  scope  or  quality  requirements. 
Otherwise,  one  increases  the  overall  risk 
level  of  the  project,  including  a  reduction 
in  quality  of  project  outcomes  [11].  The 
core  process  of  a  software  development 
project  is  developing  the  software.  It  is  not  a 
management  process,  but  an  essential 
object  process  to  be  managed  (as  depict¬ 
ed  in  Figure  2). 

Scope  management  is  best  carried  out 
by  an  independent  and  knowledgeable 
scope  manager  trained  in  ICT  project 
management,  customer  relations  (com¬ 
munication),  software  estimation,  require¬ 
ments  elicitation,  functional  size  measure¬ 
ment  (FSM),  change  management,  and 
best  practices.  As  a  third  party  usually 
hired  by  the  customer,  the  scope  manager 
is  an  advocate  to  both  the  customer(s) 
and  the  supplier(s).  The  role  is  similar  to  a 
construction  inspector/ coordinator  who 
provides  project  oversight,  governance, 
measurement,  communication,  change 
management,  progress  reporting,  and 
experience  data  collection.  The  European 
Certification  &  Qualification  Association 
formalized  the  northernSCOPE-based 
CSM  job  role  in  October  2007. 

northernSCOPE 

The  northernSCOPE  12-step  approach 
to  professional  scope  management  was 
developed  in  the  late  ’90s  by  the  Finnish 
Software  Measurement  Association 
(FiSMA)  [4];  several  CSMs  have  been 
trained  in  how  to  utilize  the  approach, 
with  4SUM  Partners  and  Quality  Plus 
Technologies  currently  leading  the  way  in 
this  training  [13]. 

The  12  steps  are  summarized  in  Table 
1 .  While  there  may  be  multiple  customers 
and/or  suppliers  (e.g,  hardware,  software, 
integration  suppliers),  the  scope  manager 
works  with  all  of  those  affected. 

Step  I:  Scope  Manager  Retained, 
Customer-Driven  High-Level 
Requirements 

The  northernSCOPE  approach  is  initiat¬ 
ed  when  the  customer  or  software  acquir¬ 
er  (or  the  software  supplier)  recognizes 


the  need  for,  and  retains,  a  scope  manag¬ 
er  for  a  new  ICT  program/  We  advocate 
using  a  CSM  to  ensure  that  there  is  a 
basic  level  of  knowledge  and  experience 
with  northernSCOPE. 

The  first  task  is  a  meeting  with  the  cus¬ 
tomer  (typically  the  program  or  project 
steering  committee)  to  outiine  the  roles 
and  responsibilities  of  the  scope  manager, 
the  customer(s),  and  the  software  suppU- 
er(s).  This  meeting  allows  the  customer  to 
ask  questions  and  to  clarify  their  role.  The 
scope  manager  also  reviews  the  high-level 
customer  requirements  for  completeness 
and  clarity,  and  discusses  their  envisaged 
scope  and  expectations  with  the  customer. 

Step  2:  Divide  Program  Into 
Subprojects 

Using  the  high-level  requirements  from 
step  1  and  the  program  subdivision  rules 
(see  Figure  3,  next  page),  the  scope  man¬ 
ager  divides  the  program  of  work  into 
appropriate  subprojects.  Note  that  this  is 


similar  to  subdividing  a  construction  pro¬ 
ject  into  distinct  subprojects,  each  of 
which  is  typically  managed  separately  and 
involves  unique  tasks.  This  is  important  to 
do  as  early  as  possible,  preferably  before 
beginning  one  or  more  software  develop¬ 
ment  projects  [14]. 

Figure  3  illustrates  the  ICT  program 
subdivision  rules,  while  Table  2  (see  next 
page)  depicts  seven  possible  (sub)project 
types.  From  the  number  of  rules  in  Figure 
3  and  the  possible  combinations,  this 
approach  leads  to  a  larger  number  of 
small  projects.  As  with  any  approach, 
there  are  a  number  of  pros  and  cons.  One 
of  the  biggest  pros  is  improved  manage¬ 
ability  so  important  to  program/  project 
success.  Neither  the  customer  nor  the 
supplier  should  resist  this  process  because 
it  divides  an  amorphous  big  bag  of  work 
into  identifiable,  manageable,  and  trace¬ 
able  parcels  of  work  that  are  easier  to 
mutually  discuss  and  scope  than  is  one 
large  monolithic  chunk.  While  not  always 


Table  1 

:  The  12  Steps  of  northernSCOPE 

step 

Concept 

Participants 

1 

Scope  manager  retained,  customer-driven  high-level  requirements. 

Customer  +  Scope  Manager 

2 

Divide  program  into  subprojects. 

Scope  Manager  +  Customer 

3 

Scope  manager  does  early  FPs  for  each  subproject  and  estimates  total  size. 

Scope  Manager 

4 

Scope  manager  and  customer  determine  and  analyze  quality  requirements. 

Scope  Manager  +  Customer 

5 

Customer  issues  request  for  proposal. 

Customer  +  Scope  Manager 
input 

6 

Customer  selects  supplier  based  on  submitted  unit  cost  per  FP. 

Customer  +  Scope  Manager 
input  +  Supplier 

7 

Requirements  specification  developed. 

Supplier  +  Customer 

8 

Scope  manager  baselines  software  size  (in  FPs)  and  product  development. 

Scope  Manager 

9 

Scope  manager  sizes  project  changes  and  cost  impact  is  evaluated. 

Scope  Manager  +  Supplier 
input 

10 

Scope  manager  quantifies  progress. 

Scope  Manager 

11 

Project  finishes  and  customer  pays  supplier  based  on  FP  delivered. 

Customer  +  Supplier 

12 

Experience  data  collected  and  stored. 

Scope  Manager 
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1 .  If  the  program  consists  of  ICT  development  and  other  development  work,  such  as  manual  process 
development,  re-organizing  staff,  or  technical  development,  different  types  of  work  should  be 
assigned  to  separate  projects. 

2.  If  you  apply  an  incremental  or  iterative  development  approach,  every  increment  or  iteration  should  be 
assigned  to  separate  projects. 

3.  Different  types  of  ICT  development  work  should  be  assigned  to  separate  projects. 

4.  If  the  program  must  be  stopped  consciously  for  a  long  time  (i.e.,  to  wait  for  external  decisions),  the 
work  before  and  after  the  break  should  be  assigned  to  separate  projects. 

5.  If  two  parts  of  either  product  or  service  development  are  of  a  similar  ICT  project  type  but  differ  from 
each  other  in  the  following  ways,  they  should  be  assigned  to  separate  projects: 

•  Development  technology 

•  Development  environment 

•  Development  team  experience 

•  Quality  requirements  of  target  result 

•  Stakeholder  dependencies 

•  Risk  level 

Figure  3:  Guidance  on  Dividing  ICT  Programs  Into  Projects  and  Subprojects  [15] 


the  case,  business  projects  where  software 
is  developed  are  typically  more  correctly 
referred  to  as  programs  [12], 

Consider  an  analogy:  A  buyer  of  goods 
requests  a  fixed  price  to  purchase  a  series 
of  items — as  many  as  can  fit  into  a  single 
standard-size  shopping  bag.  Without 
specifically  knowing  details  of  the  objects 
they  wish  to  purchase,  the  constraints  are 
the  price  (as  many  objects  as  possible  that 
can  fit),  and  that  the  objects  must  be  able 
to  be  combined  to  provide  an  easy-to-pre- 
pare  dinner  for  several  people.  As  such,  by 
discussion  and  division  of  what  is  needed 
(e.g,  cooking  utensils,  ingredients,  food 
preferences,  etc.),  both  the  buyer  and  the 
supplier  can  begin  to  better  scope  out  the 
work  in  advance.  Of  course,  when  the  sup- 
plier(s)  are  chosen  for  the  work,  then  the 
job  of  planning  each  piece  can  be  done 
(the  requirements  phase  in  software  devel¬ 
opment)  and  the  contents  of  the  shopping 


bag  are  agreed  upon. 

This  step  is  a  successful  component  of 
on-time  and  on-cost  delivery  in  northern- 
SCOPE'*.  The  goal  here  is  to  create  more 
manageable  and  definable  parcels  of  work 
that  can  be  more  readily  digested  and 
understood  by  the  business  customer; 
however,  it  is  not  the  intent  to  create 
miniscule  projects  either. 

Table  2  lists  seven  distinct  ICT  project 
types.  While  there  are  many  additional 
variations  on  these  seven  project  types, 
these  were  the  most  common  in  practice 
based  on  the  original  work  done  with  tele¬ 
com  and  software  industry  companies  in 
Finland.  See  [1 5]  for  further  details. 

The  software  development  life  cycle 
phases — such  as  requirements  through  to 
implementation — are  not  considered  as 
being  independent  ICT  project  types,  but 
rather  as  phases  within  each  subproject 
itself 


Step  3:  Scope  Manager  Does  Early 
Function  Points  (FPs)  for  Each 
Subproject  and  Estimates  Total  Size 

Using  the  high-level  customer  requirements 
for  each  subproject  from  step  2,  the  scope 
manager  performs  an  initial  function-size 
estimate  (where  appropriate).  Note  that  the 
function  size  measures  only  the  functional¬ 
ity  of  the  software;  equally  important  are 
the  quality  and  technical  requirements  (as 
outlined  in  step  4).  The  function  size  of 
the  program  software  is  the  sum  of  all  of 
its  subprojects.  Currently,  there  are  five  dif¬ 
ferent  ISO/IEC  standard  methods  for 
FSM.  The  most  commonly  used  method 
within  the  northernSCOPE  concept  is 
FiSMA  1.1  (ISO/IEC  29881),  but  all  the 
other  FSM  methods  can  be  equally  applied. 
However,  it  is  important  to  specify  the 
used  method  when  publishing  the  mea¬ 
surement  results  as  the  same  method 
should  be  appUed  throughout  the  program. 
AH  five  FSM  standards  are  detailed  in  [1 6] . 

Depending  on  the  completeness  of  the 
functional  user  requirements  (remember 
that  this  is  prior  to  a  supplier  engagement), 
this  estimated  function  size  gives  the  scope 
manager  and  the  customer  a  ballpark  idea 
of  how  big  the  ICT  program  could  be.  If 
the  functional  requirements  are  too  vague, 
then  the  customer  must  work  to  at  least 
identify  the  business  processes  to  be  sup¬ 
ported  by  the  software.  If  a  customer  can¬ 
not  state  their  needs,  then  a  supplier  cannot 
provide  an  appropriate  solution — especial¬ 
ly  before  a  request  for  proposal  (to  engage 
a  supplier  to  do  work)  is  issued.  Even  when 
a  supplier  is  contracted  to  perform  the 
requirements  elicitation  work  through  to 
software  development,  a  customer  may  not 
be  able  to  fuUy  articulate  their  needs  with¬ 
out  the  assistance  of  such  suppliers. 
However,  it  is  always  up  to  the  customer  to 
make  the  final  decision  about  what  they 
need  and  what  they  are  willing  to  pay  to 
satisfy  those  needs. 

Several  subproject  types  (and  also  work 
such  as  support  and  fixes)  are  inappropriate 
for  sizing  with  FSM.  These  include  techni¬ 
cal  upgrades,  maintenance  work,  etc.  FPs 
are  a  square  foot  type  of  measurement  used 
to  quantify  the  functional  user  requirements 
for  software;  without  them,  the  work  effort 
must  be  estimated  using  another  method, 
such  as  an  hourly  rate  or  historically  based 
estimate.  It  is  important  to  communicate  to 
the  customer  about  which  subprojects  can 
and  cannot  be  sized  using  FSM. 

Step  4:  Scope  Manager  and 
Customer  Determine  and  Analyze 
Quality  Requirements 

This  step  is  unique  to  northernSCOPE 
and  examines  the  quality  requirements  for 


Table  2:  ICT  Project  Types  [15] 


ICT  (sub)Project  Type  Classification 

Description 

1 .  Customer-specific  new  development  project 

Creates  a  completely  new  customer-specific  piece 
of  software. 

2.  Software  product  new  development  project 

Creates  a  new  software  product.  A  software  product  is 
always  developed  to  be  used  by  more  than  one 
customer.  A  software  product  may  either  be 
standalone-packaged  software  or  an  embedded  part 
of  another  product. 

3.  Software  version  enhancement  project 

Creates  a  new  version  of  existing  software.  The 
existing  software  may  be  either  customer-specific 
software  or  a  software  product. 

4.  ICT  service  development  project 

Creates  a  contract-based  continuous  or  temporary 

ICT  service.  The  service  may  be,  for  example,  either 
software  or  hardware  related,  and  consists  of 
maintenance,  support,  help  desk,  or  operating  service. 

5.  Package  software  configuration  project 

The  result  of  this  project  is  an  installed,  parameterized, 
and  user-configured  software  package. 

6.  Data  conversion  project 

A  project  where  data  is  moved  from  persistent  data 
storage  of  one  information  system  to  persistent  data 
storage  of  another  information  system.  The  software 
developed  in  a  data  conversion  project  is  often 
throw  away  in  that  it  is  only  used  once.  Even  so,  the 
pieces  of  conversion  software  may  reside  on  one  or 
more  hardware  platforms. 

7.  Software  integration  development  project 

Creates  software  that  provides  interface  services 
between  two  or  more  information  systems. 
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Figure  4:  Cost  of  southernSCOPE  Projects  Compared  to  Traditional  IT  Projects  [15] 


each  subproject  based  on  the  ISO/IEC 
9126  quality  model.  It  is  well-established 
that  the  non-functional  requirements  for 
software  can  dramatically  increase  the 
work  effort  and  cost  of  software  develop¬ 
ment;  however,  these  requirements  are 
often  not  identified  until  too  late  in  the 
project  to  respond  effectively.  According 
to  Barry  Boehm:  “A  tiny  change  in  NFR 
[non-functional  requirements]  can  cause  a 
huge  change  in  the  cost.”  [17].  He  went  on 
to  cite  the  tripling  of  a  $10  million  project 
to  $30  million  when  the  response  time  (of 
an  NFR)  went  from  four  seconds  to  one. 

Step  5;  Customer  Issues  Request  for 
Proposal 

In  this  step,  the  customer — with  the  assis¬ 
tance  of  the  scope  manager — prepares 
the  request  for  proposal  and  evaluation 
criteria  for  the  program  and  then  issues  it 
to  a  set  of  software  suppliers.  The  laundry 
lists  of  the  high-level  functional  and  quali¬ 
ty  requirements  (apportioned  by  subpro¬ 
ject)  are  included  as  attachments  to  the 
request  for  proposal  in  order  for  suppliers 
to  submit  unit  pricing  estimates. 

Step  6:  Customer  Selects  Supplier 
Based  on  Submitted  Unit  Cost  per  FP 

The  scope  manager  assists  the  customer 
to  evaluate  the  supplier  responses  and 
provides  high-level  input  regarding  the 
validity  of  their  rates  (based  on  industry 
rates).  The  customer  selects  one  or  more 
suppliers  to  meet  the  program  needs. 
More  specifically — for  those  parts  of  the 
work  where  the  development  or  enhance¬ 
ment  of  software  functions  are 
involved — the  pricing  is  provided  by  the 
supplier(s)  in  dollars  per  FR  For  other 
parts,  such  as  migration  or  operational 
upgrades,  the  pricing  is  provided  in  dollars 
per  hour  (or  other  appropriate  units).  The 
scope  manager  can  provide  a  reasonability 
gauge  for  the  cost  per  FP  values  by  com¬ 
paring  the  submitted  rates  using  published 
project  delivery  rates  in  hours  per  FP 
(converted  to  dollars  per  hour). 

Step  7;  Requirements  Specification 
Developed 

This  is  the  first  step  in  northernSCOPE 
where  the  scope  manager  does  not  play  an 
active  part.  The  customer  works  directly 
with  the  supplier(s)  to  develop  and  flesh  out 
the  requirements  for  the  subprojects. 

Step  8:  Scope  Manager  Baselines 
Software  Size  (in  FPs)  and  Product 
Development 

The  requirements  documents  for  those 
subprojects  for  which  functional  size  is 
appropriate  are  the  input  for  this  step.  The 


scope  manager  reviews  the  requirements 
and  measures  the  baseline  functional  size 
in  FP  for  each.  This  step  is  similar  to  final¬ 
izing  the  size  of  a  floor  plan  for  a  build- 
ing\  and  becomes  the  base  against  which 
any  changes  and  progress  are  tracked. 

Step  9:  Scope  Manager  Sizes 
Project  Changes  and  Cost  Impact 
Is  Evaluated 

As  changes  are  proposed  for  the  project — 
by  either  the  customer  or  the  supplier  in 
agreement  with  the  customer — the  scope 
manager  collects  and  then  records  the 
data.  A  cost  estimate  is  made  of  the  cost 
and  schedule  impact  of  such  changes  at 
the  point  in  the  project  where  they  are 
proposed  and  accepted  (based  on  the  unit 
cost[s]  originally  quoted  by  the  supplier). 
A  formal  change  management  process 
facilitates  this  step. 

Step  1 0:  Scope  Manager  Quantifies 
Progress^ 

As  each  project  progresses,  the  scope 
manager  receives  documentation  from  the 
supplier(s).  Using  the  baseline(s)  from 
step  8,  the  scope  manager  records  the 
progress  and  prepares  a  formal  status 
report  for  the  customer.  Usually  this 
occurs  on  a  monthly  basis  or  on  a  sprint 
basis  in  Agile  development,  but  the  exact 
time  frame  of  this  reporting  is  established 
with  the  parties  after  step  7. 

Step  1 1:  Project  Finishes  and 
Customer  Pays  Supplier  Based  on 
FP  Delivered 

This  is  the  second  and  final  step  of 
northernSCOPE  where  the  scope  manag¬ 
er  is  not  directly  involved.  The  customer 
pays  the  supplier(s)  based  on  the  FP  deliv¬ 
ered  for  each  subproject  and  on  whatever 
other  mechanism  where  units  of  payment 
are  used  for  the  non-FP  countable  sub- 


pro  ject(s).  From  the  customer  and  suppli¬ 
er  points  of  view,  the  program  of  work  is 
now  complete  and  the  project  manager 
closes  the  project(s). 

Step  1 2:  Experience  Data  Collected 
and  Stored'' 

The  scope  manager  finalizes  the  project 
by  recording  the  data  collected  during  and 
at  completion  of  the  project(s).  Actual 
values  for  work  effort  and  related  project 
variables  are  recorded  along  with  relevant 
project  attributes. 

It  is  worth  noting  that  there  are  sever¬ 
al  concepts  introduced  in  the  scope  man¬ 
agement  processes  that  are  not  traditional¬ 
ly  included  in  ICT  projects.  These  include: 

•  Analysis  and  classification  of  require¬ 
ments  into  independently  managed 
projects  (or  subprojects). 

•  Functional  size  measurement  of  the 
software  requirements  document  and 
project  scope. 

•  Baselining  the  project  metrics. 

•  Estimating  the  project  effort,  duration, 
and  cost  based  on  historical  project  actu¬ 
al  values. 

•  A  feedback  loop  to  estimate — and 
then  incorporate  and  track — accepted 
changes  into  the  existing  project  docu¬ 
ments. 

Through  careful  attention  to  project 
scoping  and  its  management  throughout 
the  project,  customers  and  suppliers  alike 
can  better  specify,  build,  and  acquire  quali¬ 
ty  software  products.  After  the  completion 
of  the  entire  project  (i.e.,  after  all  steps  of 
southernSCOPE  are  completed),  then  the 
results  are  tallied  and  the  overall  project  is 
compared  to  those  projects  where  no 
scope  management  was  involved.  Since  the 
introduction  of  the  concept,  practitioners 
involved  in  southernSCOPE  report  sub¬ 
stantially  lowered  costs  per  software  func¬ 
tional  size  (Figure  4). 
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Coming  Events 


March  1-4 

Ground  System  Architecture  Workshop 
Manhattan  Beach,  CA 

http://csse.usc.edu/gsaw 

March  3-5 

Secure  IT  Conference  2010 
Los  Angeles,  CA 

www.secureitconf.com 

March  9-12 

23"‘  Annual  lEEE-CS  Conference  on 
Software  Engineering  Education  and 
Training 
Pittsburgh,  PA 

http://cseet20 1 0.dei. uc.pt 

March  23-24 

GovSedU.S.  Law  2010 
The  Government  Security  Expo  & 
Conference/U.S.  Law  Conference  & 
Exposition 
Washington,  D.C. 
http://govsecinfo.com/events 
/govsec-2010 

April  12-14 

7'*  Annual  International  Conference  on 
Information  Technology 
Las  Vegas,  NV 

www.cis.gvsu. eduZ—nandigEj/Software 
EngineeringEducationTrack.html 

April  26-29 

22d  Annual  Systems  and  Software 
Technology  Conference 

%^ystems  Software 
Technology  Conference 

Salt  Lake  City,  UT 

www.sstc-online.org 

April  27-30 

MARTS  2010: 

The  Capacity  Assurance  Conference 
Chicago,  IL 

www.martsconference.com 


Coming  Events:  Please  submit  coming  events  that 
are  of  Interest  to  our  readers  at  least  90  days 
before  registration.  E-mail  announcements  to: 
<marek.steed.ctr@hill.af.mil>. 


How  to  Apply  Solid  Scope 
Management  for  Success  on 
ICT  Projects 

The  five  processes  involved  in  the 
FiSMA  Scope  Management  concept 
(Figure  2)  are  integrated  concepts;  how¬ 
ever,  all  steps  are  not  mandatory  on 
every  ICT  project.  The  initiation  and 
estimation  steps  are  prerequisites  for  the 
other  three  components,  and  these  sub¬ 
sequent  steps  are  independent  of  each 
other.  FiSMA  recommends  that  organi¬ 
zations  at  least  examine  the  last  three 
steps  for  their  applicability,  but  under¬ 
stands  exceptions  and  lighter  application 
needs  exist  [11].  Organizations  that  will 
benefit  most  from  northernSCOPE  are 
those  representing  business  areas  such 
as  banking,  insurance,  and  public  admin¬ 
istration  because  they  are  routinely 
involved  in  software  acquisition  and 
procurement.  In  these  organizations,  a 
core  business  is  information  manage¬ 
ment,  and  the  business  development 
centers  on  developing  information  sys¬ 
tems  and  software.  As  such,  traditional 
project  management  practices  have 
proven  to  be  insufficient  for  ongoing 
project  governance  because  scope  man¬ 
agement  is  not  considered  until  there  is 
a  project  already  underway.  Even  then, 
the  project  manager  has  more  critical 
tasks  to  manage  than  those  related  to 
scope. 

In  addition  to  software  acquirers, 
professional  software  suppliers  are  in 
need  of  organizational-level  processes 
to  support  continuous  process  improve¬ 
ment  and  organizational  learning.  All 
supplier  organizations  could  benefit 
from  applying  the  northernSCOPE 
processes  at  both  the  project  and  orga¬ 
nizational  levels.  Can  every  company 
gain  from  implementing  this  concept? 
Our  experience  bears  out  that  there  are 
some  small-  and  medium-sized  suppliers 
whose  process  maturity  is  considered  to 
be  ad-hoc  or  initial,  and  would  be  better 
served  by  first  concentrating  on  devel¬ 
oping  or  improving  core  processes  such 
as  time  recording  and  invoicing. 
Nonetheless,  the  division  of  projects 
into  software  projects  to  be  indepen¬ 
dently  worked  can  assist  even  the  most 
disorganized  or  immature  organizations 
in  improving  ICT  project  management. 

Through  international  collaboration 
and  consulting,  we  have  found  that 
northernSCOPE  (and  for  that  matter, 
software  process  improvement)  just 
doesn’t  always  succeed  even  with  the 
best  of  intentions.  Reasons  for  failure  or 
partial  adoption  of  sound  ideas  such  as 


scope  management  can  run  the  gamut 
between  lack  of  support  or  understand¬ 
ing  to  internal  sabotage  by  organizations 
weary  from  whiplash  changes  imposed  by 
management.  It  is  useful  to  consider  the 
critical  success  factors  of  process 
improvement  before  launching  new  con¬ 
cepts.  Sources  such  as  [18,  19,  20,  21] 
outline  these  critical  success  factors. 

Conclusion 

Through  the  results  of  northernSCOPE 
and  southernSCOPE,  scope  manage¬ 
ment  processes  are  a  proven  means  of 
leveraging  and  augmenting  professional 
project  management  on  ICT  projects. 
With  the  current  levels  of  project 
rework  in  the  vicinity  of  45  percent  of 
development  effort,  our  industry  surely 
needs  to  increase  its  ICT  project  success, 
and  one  proven  way  was  expounded  on 
in  this  article. 

ICT  program  recovery  can  greatly 
benefit  from  a  solid  12-step  program, 
starting  with  an  admittance  of  the  facts, 
determination,  trust,  continuous  con¬ 
trol,  and  professional  guidance  and  sup¬ 
port  from  a  scope  manager.^ 
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Notes 

1.  More  about  successful  use  of 
northernSCOPE  can  also  be  found  in 
[5]  and  [6]  while  more  on  southern- 


SCOPE  can  be  found  in  [7]  and  [8]. 

2.  Scope  management  is  put  in  the  mid¬ 
dle  of  Figure  1,  though  [12]  introduces 
knowledge  areas  in  a  different  order 
originally.  The  order  is  not  important, 
but  here  we  want  to  emphasize  the 
central  role  of  scope  management  in 
software  development. 

3.  At  this  point,  customers  may  refer  to 
this  as  the  project;  however,  it  is  more 


Ukely  an  ICT  program  with  several 
projects.  See  step  2. 

4.  This  step  is  not  a  part  of  southern- 
SCOPE. 

5.  When  a  floor  plan  is  initiaUzed  at  the 
beginning  of  a  home  construction 
project,  it  becomes  the  master  plan 
that  the  built  area  in  square  feet  or 
square  meters  and  any  changes  to  the 
floor  plan  are  measured  against. 
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Stealth  CPI:  Managing  Work  Products  to 
Achieve  Continuous  Process  Improvement 

Ron  Abler  and  Ted  Warren 
NAVAIR 


Whik  identifying,  defining,  and  organising  activities  into  graphical  process  diagrams  is  helpful  during  Continuous  Process 
Improvement  (CPI)  planning,  it’s  not  very  useful  to  our  managers  and  their  teams  during  impkmentation.  Most  managers 
understand — and  therefore  manage — through  work  products  (WPs).  Managing  WPs  has  proven  to  he  simpkr  and  more 
intuitive  than  managing  processes,  dealing  directly  with  the  endpro ducts  themselves.  Stealth  CPI  centers  on  existing  WPs, 
permitting  organisations  to  implement  process  improvement  (PI)  incrementally,  purposefully — and  even  stealthily. 


Most  if  not  all  PI  standards,  models, 
and  methodologies  concern  them¬ 
selves  with  processes  during  the  planning 
and  implementation  phases.  But  when  it 
comes  to  the  evaluation  phase,  evaluators 
do  not  look  at  processes  but  at  the  con¬ 
crete  evidence  (artifacts  and  WPs)  show¬ 
ing  that  the  processes  have  been  fol¬ 
lowed.  Nevertheless,  PI  remains  an 
abstract  exercise  that  is  not  intuitively 
obvious  to  the  majority  of  technical  pro¬ 
fessionals  who  are  not  specialists  in  the 
PI  field.  This  fact  has  led  to  a  myriad  of 
models  and  methodologies  that  attempt 
to  simplify  the  concept  of  PI.  The  end 
result  has  been  the  creation  of  an  exten¬ 
sive  consulting  industry  dedicated  to 
shepherding  organizations  through  the 
confusing  maze  and  jargon  of  formal 
standards  compliance. 

In  our  organization,  the  Presidential 
Helicopters  Program  In-Service  Inte¬ 
grated  Product  Team  (VH-IS),  we  are 
prototyping  a  concept  that  we  call  Stealth 
CPI.  Our  goal  is  to  improve  our  process¬ 
es  by  structuring  our  WPs  so  that  our 
people  can  be  best  practices-  and  CMMI- 
comphant  just  by  doing  their  jobs.  This 
concept  is  based  on  lessons  learned  from 
previous  implementations  of  CMMI  in 
other  organizations: 

1 .  Process  diagrams  are  abstract  and  are, 
at  best,  graphic  representations  of 
activities  conducted  within  and  in 
support  of  the  described  process. 
Thus,  process  diagrams  are  tools,  not 
products. 

2.  Processes  are  made  up  of  activities, 
and  each  activity  results  in  one  or 
more  WPs.  Only  the  WP,  not  the  gen¬ 
erating  activity,  can  transfer  the  activ¬ 
ity’s  productivity  elsewhere  within  an 
organization.  Any  activity  or  suite  of 
related  activities  that  does  not  pro¬ 
duce  a  desirable  WP  is  held  suspect 
because  it  may  be  unnecessary  or 
unproductive. 

We  feel  that  neither  processes  nor 
activities  alone  benefit  organizations. 


With  this  in  mind,  our  plan  implements 
CPI  by  managing  WPs  instead  of 
processes.  Stealth  CPI  meets  its  PI  goals 
in  three  phases: 

1.  Planning  Phase 

•  Documenting  and  refining  the 
visual  representation  of  the  im¬ 
proved  processes. 

•  Mapping  WPs  to  activities. 

•  Developing  work  folders  (aU  con¬ 
taining  guidance)  to  support  im¬ 
proved  WPs. 

2.  Implementation  Phase 

•  Publishing  the  work  folders. 

•  Assigning  the  improved  WPs. 

3.  Assessment  Phase 

•  Conducting  continuous  assess¬ 
ments.  For  example,  SCAMPI  B 
and  C  assessments  in  a  CMMI  en¬ 
vironment. 

Project  managers  are  a  breed  apart  in 
American  business  and  government. 
They  walk  a  narrow  line  defined  by 
requirements,  schedule,  budget,  and 
resources,  complicated  by  the  demands 
of  superiors  and  the  needs  of  subordi¬ 
nates.  Happy  is  the  manager  who  knows 
exactly  what  WPs  are  required  and  when, 
and  who  has  been  given  all  the  resources 
necessary  to  produce  those  WPs  correct¬ 
ly,  completely,  and  in  a  timely  manner.  In 
fact,  the  same  thing  can  probably  be  said 
of  most  employees  in  any  organization, 
regardless  of  the  organization’s  business. 

When  the  specter  of  PI  rears  its 
intimidating  head  (as  it  will  eventually  in 
every  competitive  business),  all  too  fre¬ 
quently  the  conventional  response  is  to 
start  with  process  definition  in  the  form 
of  process  diagrams.  In  too  many  cases, 
the  end  result  of  this  classic  approach  is 
improved  process  diagrams  instead  of 
improved  processes.  Giving  project  man¬ 
agers  a  process  diagram  and  asking  them 
to  make  the  contained  improvements  is 
usually  a  waste  of  their  time.  Training 
project  managers  to  become  PI  special¬ 
ists — in  the  hope  that  the  worker  bees  will 
fulfill  the  desires  of  the  management 


queens  and  drones — can  be  expensive 
and  wasteful.  We  believe  it  is  more  effi¬ 
cient  and  cost-effective  to  implement  PI 
by: 

•  Rolling  out  improved  WP  tool  sets 
rather  than  improved  process  dia¬ 
grams. 

•  Training  our  people  to  be  experts  in 
our  WPs  rather  than  in  PI. 

And  our  people  agree. 

Stealth  CPI  is  a  bottoms-up  approach 
that  takes  advantage  of  a  simple  fact:  It 
is  far  simpler  and  infinitely  more  intuitive 
to  identify  and  improve  WPs  than  it  is 
with  processes.  It  is  more  conceivable  to 
envision  a  successful  organization  with¬ 
out  a  single  process  diagram  than  it  is  to 
imagine  one  without  WPs.  Even  the 
clearest  diagram  of  the  best  possible 
process  can’t  single-handedly  improve  an 
existing  process.  This  is  where  PI  models 
and  methodologies  normally  enter  the 
picture:  to  show  us  where  to  go  next. 
Regardless  of  the  standard  to  be  used,  all 
of  them  start  with  processes  but  end  with 
WPs.  The  reason  for  this  is  that  assessing 
the  goodness  of  a  process  is  neither  fea¬ 
sible  nor  objective.  Take  the  following 
exchange  as  a  typical  example: 

Evaluator:  “Do  you  conduct  peer 
reviews?”  (If  peer  reviews  are  part  of 
the  standard  being  evaluated,  then 
“Yes”  is  the  only  correct  answer  to 
this  question). 

Evaluee:  “Yes.” 

Evaluator:  “Prove  it!” 

At  this  point,  it  is  neither  correct  nor 
sufficient  to  reply:  “Our  next  peer  review 
is  in  two  weeks.  Come  back  then  and 
watch.” 

In  other  words,  the  proof  lies  not  in 
the  process,  but  in  the  artifacts  and  WPs 
that  result  from  the  process.  Evaluators 
routinely  presume  that  the  existence  of  a 
desired  artifact  proves  that  its  generating 
activity  did,  in  fact,  take  place.  The  quali¬ 
ty  of  the  artifact  is  presumed  to  reflect 
the  quality  of  its  generating  activity. 

NAVAIR  has  developed  its  “System 
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Engineering  Guide,”  which  mandates  a 
process  of  System  Engineering  Technical 
Reviews  (SETRs)  [1],  The  SETR  entry 
and  exit  criteria  are  more  easily  under¬ 
stood  when  they  are  described  as  WPs 
than  when  they  are  portrayed  as  activities. 
Thus,  a  standardized  report  containing 
the  output  of  the  review  activities  usually 
makes  a  more  useful  exit  criterion  than  a 
statement  that  the  reviews  activities  were 
completed. 

Since  evaluations  are  always  conduct¬ 
ed  at  the  artifact  level,  it  makes  sense  to 
approach  PI  from  the  same  perspective. 
PI  constitutes  a  chain  with  connected 
links  from  the  envisioned  process 
through  the  implementing  activities  right 
down  to  the  artifacts  that  constitute  the 
WPs.  It  is  easier,  more  intuitive,  and  more 
operationally  effective  to  pull  the  chain 
by  improving  real-world  WPs  than  it  is  to 
push  the  chain  by  improving  abstract 
processes. 

The  Capability  Waypoint 
Matrix 

Robert  Jacob,  the  Head  of  the  Aviation 
Safety  Department  at  the  Atlantic  Test 
Range  at  the  Naval  Air  Station  (NAS) 
Patuxent  River,  and  Ron  Abler  developed 
the  concept  of  displaying  and  tracking 
progress  in  the  department’s  implementa¬ 
tion  of  CMMI  with  a  two-dimensional 
matrix,  called  a  Capability  Waypoint 
Matrix  (CWM).  Abler  presented  the 
CWM  at  a  Software  Engineering  Process 
Group  (SEPG)  [2]  meeting  where  the 
SEI  had  solicited  ideas  for  simplifying  the 
CMMI.  A  panel  of  CMMI  sponsors. 


CMMI  instructors,  and  SCAMPI  leads 
peer-reviewed  the  CWM  and  declared  it 
to  be  CMMI-compliant. 

The  Aviation  Safety  Department 
rolled  out  their  CWM  in  October,  2007, 
and  in  2008  published  their  T-Risk  model 

[3] ,  which  was  the  first  known  implemen¬ 
tation  of  CMMI  in  an  operational  avia¬ 
tion  safety  environment. 

As  the  term  matrix  implies,  a  CWM  is 
simply  an  array  of  rows  and  columns  (see 
Figure  1).  Each  cell  that  is  defined  by  the 
intersection  of  a  single  row  and  column  is 
called  a  waypoint.  The  CWM  is  a  snapshot 
of  the  work-product  improvement  status 
at  a  specific  moment  in  time.  In  aviation,  a 
waypoint  is  a  point  in  physical  space 
defined  by  latitude,  longitude,  altitude,  and 
time  of  passage.  In  the  CWM,  a  waypoint 
is  defined  by  process  name,  effectiveness, 
efficiency,  and  date/time  of  the  snapshot 
(i.e.,  evaluation).  The  rows  and  columns 
arrange  the  WPs  in  the  structure  defined 
by  the  CPI  model  in  use. 

While  it  is  true  that  the  CWM  can  be 
used  to  support  any  PI  methodology  that 
has  artifacts  (or  WPs),  the  following 
description  is  based  on  the  CMMI  model 

[4] ,  which  is  the  methodology  that  VH-IS 
uses. 

A  CWM  is  generated  as  a  spreadsheet 
for  each  CMMI  process  area  in  the  orga¬ 
nization’s  profile.  Figure  1  depicts  a  sim¬ 
plified  CWM  for  the  risk  management 
process  area.  The  Y-axis  (represented  by 
the  rows)  charts  overall  effectiveness,  and 
the  X-axis  (represented  by  the  columns) 
charts  overall  efficiency.  In  CMMI,  effec¬ 
tiveness  (defined  as  simple  performance) 


is  defined  by  the  achievement  of  specific 
goals  (SGs).  Efficiency  (i.e.,  better,  faster, 
cheaper,  smarter)  is  defined  by  fulfillment 
of  the  generic  goals.  Therefore,  in  the 
CWM,  the  rows  are  populated  by  specific 
practices  (SPs)  and  the  columns  by  the 
generic  practices  (GPs).  Thus,  columns  A 
and  B  of  this  CWM  hold  the  risk  man¬ 
agement  SGs  and  practices,  while  rows  D 
through  P  hold  the  generic  practices. 
Column  C  holds  the  WP  (i.e.,  direct  arti¬ 
fact)  that  results  from  each  specific  prac¬ 
tice  (these  are  what  the  SCAMPI  team 
looks  for).  Since  Stealth  CPI  tracks  and 
manages  WPs  rather  than  processes,  it  is 
important  that  each  row  hold  only  one 
WP.  If  a  SP  generates  more  than  one  WP, 
add  a  row  or  rows  as  necessary  to  accom¬ 
modate  them.  WPs  tend  to  be  organiza¬ 
tion-specific,  so  Figure  1  refers  to  them 
genetically  as  WP  1.1  through  3.2. 

Examples  of  Risk  Management  WPs 
might  be  a  risk  list  for  WP  2.1  and  a  fully 
categorized  and  prioritized  risk  list  for 
WP  2.2.  WP  3.1  would  obviously  be  mit¬ 
igation  plans.  If  there  are  more  than  one, 
they  can  be  designated  as  3.1a,  3.1b,  etc. 
WP  3.2  might  be  periodic  reports  that 
track  and  document  the  implementation 
of  each  mitigation  plan. 

Once  a  CWM  has  been  completed  to 
this  point  (filled  in  with  SPs,  GPs,  and 
WPs),  the  simple  secret  of  Stealth  CPI 
becomes  apparent.  It  is  so  simple  that  it 
sounds  trivial.  All  that  is  required  to 
achieve  PI  is  to  apply  each  GP,  one  at  a 
time,  to  each  WP  and  track  the  status  of 
so  doing  in  each  waypoint.  That  status 
can  be  represented  textually  (e.g.,  excel- 


Figure  1:  Sample  Capability  Waypoint  Matrix  (for  the  Risk  Management  Vrocess  Area) 
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lent,  good,  inadequate),  numerically  (e.g., 
1  through  10),  or  graphically  (e.g.,  green, 
yellow,  and  red),  as  desired.  The  CWM  in 
Figure  1  uses  colors  to  denote  status  in  a 
manner  similar  to  the  SCAMPI. 

The  technique  of  Stealth  CPI  starts 
with  the  waypoints  in  column  D.  If  it  can 
be  demonstrated  that  a  WP  simply  exists 
and  that  its  indirect  artifacts  (e.g.,  memos, 
meeting  minutes,  emails,  etc.)  demon¬ 
strate  that  it  is  in  regular  use,  then  its  way- 
point  in  column  D  can  be  filled  in  with  a 
satisfactory  status.  Once  all  the  SPs  in  a 
process  area  have  satisfactory  marks  in 
column  D  of  the  CWM,  it  can  be  said 
that  the  organization  is  minimally  effec¬ 
tive  in  that  process  area  (i.e..  Capability 
Level  1  “Performed”). 

Admittedly,  higher  capability  levels 
are  more  difficult  to  achieve  than  merely 
demonstrating  the  existence  of  WPs,  but 
the  Stealth  CPI  technique  remains  just  as 
simple  at  all  capability  levels:  Apply  each 
GP,  one  at  a  time,  to  each  WP  and 
improve  that  WP  as  necessary  to  comply 
with  the  GP.  Take,  for  example,  a  mitiga¬ 
tion  plan,  WP  3.1  (in  row  12,  column  C). 
The  waypoint  under  GP  2.5,  Train  the 
People  (row  12,  column  I)  would  contain 
a  satisfactory  mark  if  it  can  be  shown 
that  the  employee  (s)  who  develop  mitiga¬ 
tion  plans  are  properly  trained  to  do  so. 
Similarly,  the  waypoint  under  GP  2.6, 
Manage  Configurations  would  hold  a 
“Not  Applicable”  mark  if  the  mitigation 
plan  was  not  a  configuration  item  (Cl). 
Since  it  is  likely  that  a  mitigation  plan 
would  be  a  Cl,  the  waypoint  in  column  J 
would  hold  a  satisfactory  grade  if  the 
mitigation  plan  shows  evidence  of  con¬ 
figuration  management,  such  as  a  current 
document  version  number. 

One  can  see  that  the  CWM  tracks 
both  the  simple  existence  (effectiveness) 
and  the  goodness  of  that  WP  (efficiency) 
relative  to  the  discriminators  required  by 
whatever  standard  an  organization 
chooses  to  use  (such  as  the  GPs  in 
CMMI).  It  is  the  CWM  that  enables 
Stealth  CPI  to  achieve  PI,  one  WP  at  a 
time. 

Work  Folders 

Once  the  processes,  their  related  activi¬ 
ties,  and  the  resulting  WPs  have  been 
mapped  onto  the  CWM,  it  becomes  pos¬ 
sible  to  begin  improving  processes  by 
directing  the  desired  improvement  in  the 
organization’s  WPs. 

Take  GP  2.6,  Manage  Configurations, 
for  example:  Every  WP  (i.e.,  each  row)  in 
the  CWM  has  its  waypoint  in  the  GP  2.6 
column.  If  a  WP  is  a  configuration  item 
(i.e.,  appears  on  the  configuration  item 


list,  itself  a  WP),  then  Stealth  CPI  expects 
that  evidence  of  configuration  manage¬ 
ment  will  be  reflected  in  the  WP.  In  the 
case  of  a  recurring  report,  the  improvement 
might  be  the  simple  addition  of  a  docu¬ 
ment  version  or  control  number. 
Improvement  in  WPs  are  managed 
through  the  use  of  work  folders,  which 
completely  define  the  who,  what,  when, 
where,  why,  and  how  of  each  WP. 

The  goal  of  a  work  folder  is  to  pro¬ 
vide  staff  members  with  all  the  informa¬ 
tion  and  tools  necessary  to  produce  WPs 
of  the  desired  quality.  As  appropriate  to 
each  WP,  a  work  folder  may  contain  a 
template,  a  tailoring  guide,  criteria  for 
use,  a  process  or  activity  diagram,  com- 


*^0ur  staff  does  not 
have  to  be  specifically 
trained  in  CMMI 
or  even  conversant 
with  PI  theory, 
concentrating  on  doing 
what  they  do  best: 
generating  the 
highest-quality 
WPs  possible/* 

plete  instructions  (if  not  contained  in  the 
template  itself),  perhaps  a  PI  Indicator 
Description,  and  even  the  actual  training 
materials  (or  pointers  to  the  training)  nec¬ 
essary  for  proper  use  and  delivery.  In  a 
nutshell,  the  work  folder  completely 
defines  the  activity  or  activities  that  gen¬ 
erate  the  WP.  In  total,  the  work  folders 
implement  the  processes  of  the  organiza¬ 
tion  in  an  immediately  usable  form. 

Stealth  CPI  in  the  Program 
Office 

VH-IS  is  adopting  the  Program  Manage¬ 
ment  Office  (PMO)  approach  of  consol¬ 
idating  functions  and  standardizing 
methodologies  and  templates  for  use  in 
the  management  and  delivery  of  projects. 
This  new  approach  will  be  the  prototype 
for  Stealth  CPI.  If  successful.  Stealth  CPI 
will  be  implemented  for  new  projects  and 
eventually  retrofitted  throughout  the 
organization.  Stealth  CPI  will  be  the  basis 
for  re-use,  wherever  possible,  of  PMO- 


like  processes  and  WPs.  VH-IS  has 
selected  the  continuous  representation  of 
CMMI  with  a  tailored  profile  consisting 
of  six  high-priority  process  areas:  project 
planning,  project  monitoring  and  control, 
requirements  development,  requirements 
management,  configuration  manage¬ 
ment,  and  risk  management. 

The  first  step  was  to  generate  a  CWM 
for  each  process  area  by  listing  the  SPs  in 
the  rows,  and  the  GPs  in  the  columns. 
The  next  step  was  to  map  the  WPs  from 
NAVAIR’s  SETR  (entry  and  exit  criteria) 
to  their  related  SPs.  This  involved  adding 
some  practices  or  activities  to  account  for 
non-CMMI  WPs  required  by  the  SETR 
as  well  as  adding  some  new  WPs  expect¬ 
ed  in  the  CMMI  but  not  called  for  in  the 
SETR. 

After  each  individual  CWM  has  been 
fully  populated  (such  that  every  WP  maps 
to  a  single  practice  or  activity  and  every 
practice  or  activity  maps  to  a  unique  WP), 
the  work  folders  for  each  WP  can  be  cre¬ 
ated  and  placed  in  a  shareable  repository. 

Once  a  work  folder  has  been  com¬ 
pleted  and  made  available  to  staff  mem¬ 
bers  on  a  project-by-project  basis,  the 
VH-IS  can  track,  manage,  and  ultimately 
fulfill  all  of  the  requirements  of  the  gov¬ 
erning  SETR  in  a  CMMI-compliant  man¬ 
ner  by  simply  assigning  the  WPs  that  ful¬ 
fill  the  entry  and  exit  criteria  of  the  SETR 
(plus  supporting  CMMI-specific  WPs,  as 
necessary).  From  this  point  on,  our  ulti¬ 
mate  goal  of  constant  PI  can  be  achieved 
through  regular  WP  improvement  as  long 
as  each  improvement  is  defined  and  facil¬ 
itated  by  the  tools  in  the  concomitant 
work  folders.  Our  staff  does  not  have  to 
be  specifically  trained  in  CMMI  or  even 
conversant  with  PI  theory,  concentrating 
on  doing  what  they  do  best:  generating 
the  highest-quality  WPs  possible. 

In  summary:  Contrary  to  classic  CPI, 
we  will  employ  and  improve  our  existing 
WPs;  established  processes  and  their 
improvement  will  be  the  consequent  by¬ 
products.  The  CMM  will  serve  as  our  tool 
and  our  map. 

Early  Success  Indicators 

At  VH-IS,  we  are  on  the  verge  of  declar¬ 
ing  success  at  Capability  Level  1  for  our 
profile.  As  soon  as  we  have  mapped  an 
existing  WP  (direct  artifact)  to  each  SP 
and  can  demonstrate  their  active  use  with 
indirect  artifact(s),  we  can  claim  Level  1 
because  we  can  prove  that  we  are  per¬ 
forming  each  specific  practice. 
Considering  that,  we  will  have  reached 
Level  1  without  requiring  the  very  time- 
consuming  and  (unfortunately)  distract¬ 
ing  SEPG.  In  fact,  we  will  have  achieved 
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Software  Defense  Application 

Defense  organizations  will  find  that  Stealth  CPI  is  much  more  intuitive  and  cost- 
effective  than  traditional  CPI  because  it  approaches  the  problems  from  the  perspec¬ 
tive  of  real-world  work  products  rather  than  abstract  processes.  This  article  steps  the 
reader  through  the  set-up  and  implementation  of  Stealth  CPI  with  the  use  of  its  cen¬ 
tral  tool,  the  Capability  Waypoint  Matrix.  Stealth  CPI  saves  time  and  money  over 
more  conventional  approaches  because  it  requires  less  process-specific  training,  fewer 
people  dedicated  to  the  CPI  project  itself,  and  therefore  less  overhead  to  achieve  the 
same  results.  Better-managed  WPs  foster  better  reuse,  result  in  more  efficient  imple¬ 
mentation,  and  are  easier  to  improve  than  abstract  processes. 


this  with  only  the  part-time  involvement 
(16  hours  per  week)  of  the  standards 
engineer  and  a  couple  of  hours  per  week 
of  the  team  lead’s  time — a  definite  cost- 
avoidance.  Furthermore,  we  have  gotten 
this  far  without  requiring  any  formal 
CMMI  training  of  our  staff — a  definite 
cost-savings.  This  is  an  early  indicator 
that  we  can  continue  this  technique 
through  Levels  2  and  3,  and  we  should  be 
able  to  do  so  very  cost-effectively,  while 
imposing  little  or  no  process-improve¬ 
ment  distraction  into  the  equation. 

Also,  in  the  course  of  reaching  Level 
1,  we  will  have  mapped  our  SPs  to  the 
NAVAIR-mandated  technical  review  sys¬ 
tem,  thereby  killing  two  birds  with  one 
stone:  We  have  demonstrated  the  feasibil¬ 
ity  of  implementing  a  technical  review 
system  that  is  intrinsically  CMMI-compli- 
ant  through  common  WPs.  We  consider 
this  to  be  a  necessary  milestone  toward 
the  success  of  our  initially  stated  ultimate 
goal,  which  is  to  structure  our  WPs  so 
that  our  people  can  be  best-practices 
compliant  just  by  doing  their  jobs. 

Ten  Steps  to  Successful 
Stealth  CPI 

If  you  plan  on  starting  to  utilize  Stealth 
CPI  in  your  organization,  here  are  10 
essential  steps  to  success: 

1.  Create  a  CWM  template  for  each 
process  area'. 

2.  List  each  activity  within  the  process 
on  its  own  row  in  column  A  and  B.  In 
CMMI,  these  are  SPs. 

3.  List  the  tide  of  each  activity’s  WP  in 
column  C. 

4.  If  an  activity  generates  more  than  one 
WP,  add  a  row  for  each  WP,  repeating 
the  activity  name  in  columns  A  and  B, 
and  the  name  of  the  artifact  in  col¬ 
umn  C.  In  column  D,  insert  a  word  or 
phrase  that  will  document  the  simple 
existence  of  each  artifact. 

5.  Determine  the  discriminators  that  will 
be  used  to  evaluate  the  incremental 
improvement  of  the  process.  In 
CMMI,  these  are  the  GPs.  Place  the 
name  of  each  discriminator  in  the  top 
row  of  each  column,  beginning  with 
column  D. 

6.  For  each  WP,  assign  an  owner/man¬ 
ager  who  is  responsible  for  its  mainte¬ 
nance  and  upkeep. 

7.  Define  a  complete  work  folder  for 
each  artifact  or  group  of  artifacts  that 
represents  a  discrete  WP. 

8.  Conduct  internal  audits  (gap  analyses) 
of  the  artifacts  and  WPs,  recording 
the  status  of  each  in  the  appropriate 
waypoint,  starting  with  column  D 


(exists)  and  denoting  each  qualifying 
waypoint  under  the  discriminators,  as 
appropriate. 

9.  Use  the  results  of  the  audits  to  evalu¬ 
ate  the  overall  PI  program  and  to  con¬ 
tinue  improving  the  organization’s 
WPs. 

10.  When  all  waypoints  in  a  process  are 
satisfactory  for  the  desired  next  level 
of  improvement,  DECLARE  VIC- 
TORY14 
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CMMI,  Swiss  Cheese,  and  Pareto 


Darrell  Corbin 
Independent  Consultant 

Getting  ready  for  a  CMMI  appraisal  is  rarely  easy.  But  what  if  your  organisyition  is  virtual,  has  critical  2417  commit¬ 
ments,  and  overtime  is  the  norml  What  if  key  staff  is  overloaded  and  time  available  for  CMMI  is  minimal?  This  article 
provides  a  case  study  showing  how  these  obstacles  can  be  overcome  in  preparation  for  a  CMMI  for  Development  (CMMI- 
DEV)  Eevel 3  appraisal 


Imagine  that  you’ve  just  been  asked  to 
assume  the  role  of  process  analyst  and 
lead  the  effort  for  an  organization  to  pass 
a  CMMI-DEV  Level  3  appraisal.  Most  of 
the  40-member  staff  has  participated  in 
earlier  self  assessments  and  the  software 
developers  are  from  a  Level  5  company. 
Everyone  knows  about  CMMI  and  its 
predecessors,  primarily  the  Software 
CMM.  The  assignment  sounds  easy, 
right?  Wrong! 

While  I’ve  come  across  many  organi¬ 
zations  with  similar  challenges,  this  article 
will  focus  on  a  case  study  of  one  organi¬ 
zation — for  our  purposes.  I’ll  call  them 
Company  C — that  overcame  obstacles 
synergistically  using  the  Swiss  Cheese 
Method  and  the  Pareto  Principle. 

The  Work  Environment 

Company  C’s  IT  staff  is  mostly  virtual 
with  members  in  three  states.  The  devel¬ 
opers  (subcontractors)  are  from  India. 
The  group  is  responsible  for  key  compa¬ 
ny  software  and  must  be  available  24/7. 
Overtime  is  the  norm.  Software  upgrades 
are  frequent  and  take  priority.  Time  avail¬ 
able  for  CMMI  is  rare.  Meetings  are  held 
using  WebEx,  and  deliverables  are  stored 
on  SharePoint,  Wikis,  file  servers,  or  lap¬ 
tops.  There  is  no  Process  Asset  Library 
(PAL)  at  the  beginning. 

Company  C  has  improved  software 
processes  for  years  by  facilitating  work¬ 
shops.  Conference  rooms  are  lined  with 
flip  charts  showing  process  flows,  data 
stores,  artifacts,  and  more.  But  this 
approach  doesn’t  work  when  the  staff  is 
absent  or  virtual,  and  the  IT  staff  has 
only  a  few  minutes  available  to  help  lead¬ 
ership  at  random  times.  What  did 
Company  C  do? 

Remember  the  Swiss  Cheese 
Method? 

Alan  Lakein’s  1973  book,  “How  to  Get 
Control  of  Your  Time  and  Your  Life,” 
introduced  the  Swiss  Cheese  Method  and 
talked  about  how  we  can  nibble  away  at  a 
task  or  project.  He  suggested  that  when 
we  only  have  five  minutes  or  less,  we 


should  spend  it  on  high-priority  tasks  and 
constantly  nibble  away  at  them. 
Otherwise,  we  slide  the  schedule  and 
keep  waiting  until  we  have  enough  dme  to 
work  on  a  task. 

Project  plans  are  rarely  defined  in 
minutes  and  doing  so  isn’t  recommended. 
In  the  case  of  Company  C,  the  process 
analyst  used  the  Swiss  Cheese  Method  on 
critical  path  tasks.  The  process  analyst  did 
not  get  an  hour  of  a  functional  analyst’s 
time,  but  was  given  a  few  minutes  after 
specifically  detailing  the  help  that  was 
needed. 

By  using  the  Swiss  Cheese  Method  in 
a  variety  of  ways.  Company  C  was  able 
to: 

•  Identify  source  documents  (versus 
multiple  versions)  for  their  PAL. 
More  than  100  documents  are  put  in 
the  PAL. 

•  Obtain  comments  on  sections,  one  at 
a  time,  of  their  quality  plan  and 
dozens  of  other  process  documents. 

•  Prioritize  process  improvement  team 
actions.  The  team  completes  more 
than  100  actions  prior  to  the 
appraisal. 

•  Resolve  inconsistencies  in  documents 
under  configuration  management. 

•  Update  portions  of  the  project  plan. 

•  Resolve  dozens  of  internal  assess¬ 
ment  comments  (one  at  a  dme). 

Tools  such  as  instant  messaging 

helped  identify  these  small  blocks  of 
time,  even  as  little  as  five  minutes.  In 
effect,  this  allowed  for  the  constant  nib¬ 
bling  away  at  the  hundreds  of  mini-tasks 
required  to  get  ready  for  the  appraisal. 
The  process  analyst  worked  with  the  staff 
to  identify  these  small  time  slices  every 
day  and  progress  was  closely  measured. 

Use  of  the  Swiss  Cheese  Method 
required  the  process  analyst  to  work 
much  closer  with  the  project  manager  as 
compared  to  a  typical  project.  MS  Project 
was  used  to  develop  the  initial  schedule, 
then  Primavera  P3e  was  used  to  manage 
the  CMMI  project.  That’s  where  another 
time-management  concept,  the  Pareto 
Principle,  came  in  handy. 


ThankYoUjVilfredo  Pareto 

Many  of  us  have  used  the  80-20  rule,  the 
more  common  term  for  the  Pareto 
Principle.  Recall  that  Vilfredo  was  an  Italian 
economist  who  found  that  80  percent  of 
the  land  in  Italy  was  owned  by  20  percent  of 
the  population.  His  findings  evolved  into 
the  rule  that  says,  by  one  application:  20 
percent  of  the  time  spent  on  the  vital  few 
yields  80  percent  of  the  results — and  80 
percent  of  the  time  spent  on  the  trivial  many 
yields  only  20  percent  of  the  results. 

So  what  does  that  have  to  do  with 
CMMI?  When  Company  C’s  CMMI  effort 
was  started,  there  was  no  project  plan,  no 
PAL,  few  completed  process  documents, 
and  multiple  versions  of  key  documents. 
And  there  was  no  common  understanding 
of  the  definition  of  aU  163  key  areas  (22 
process  areas,  three  defined  by  Company 
C).  From  a  Pareto  view,  80  percent  of  the 
time  spent  on  process  activities  only  yield¬ 
ed  20  percent  of  the  target:  100  percent 
CMMI  compliance. 

Prior  to  the  CMMI  project,  there  was 
Uttle  evidence — by  any  measure — that  the 
time  was  being  well-spent.  Vilfredo  would 
be  happy  to  know  that  Company  C  used 
his  principle  to  help  them  focus  on  the  vital 
few  to  obtain  the  largest  portion,  the  80  per¬ 
cent,  of  the  positive  results. 

The  process  analyst  and  project  manag¬ 
er  reviewed  the  CMMI  statement  of  work 
and  associated  work  breakdown  structure 
and  schedules,  and  identified  tasks  in  the 
vital few  category,  including: 

•  Updating  the  quality  plan.  This  turned 
out  to  be  a  major  task. 

•  Identifying  metrics  for  each  process 
area:  to  give  proof  that  use  of  CMMI 
was  yielding  positive,  measurable  results 
in  terms  of  cost,  quality,  and  schedule. 

•  Establishing  a  quality  audit  program 
and  conducting  quality  audits. 

•  Training  for  all  Level  3  and  some  Level 
2  Process  Areas  (presented  in  one-hour 
segments  via  WebEx). 

•  Updating  self-assessment  by  using  a 
proprietary  spreadsheet  for  the  staff  to 
record  their  personal  assessments  by 
key  area. 
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CMMI,  Swiss  Cheese,  and  Pareto 


Software  Defense  Application 

The  article  describes  some  creative  techniques  defense  organizations  can  use  to 
achieve  100  percent  CMMI  compliance  in  a  virtual  environment.  This  case  study  of 
a  major  contractor  describes  how  the  organization  prepared  for  a  CMMI-DEV 
Level  3  appraisal.  Two  of  the  techniques  come  from  basic  time  management  tenets: 
the  Swiss  Cheese  Method  and  the  Pareto  Principle.  Other  lessons  learned  addressed 
the  process  improvement  team,  configuration  management,  and  communications 
techniques. 


Synergy  of  Swiss  Cheese  and 
Pareto 

Tasks  are  constantly  reprioritized  in  any 
big  project,  but  determining  which  tasks 
are  part  of  the  vital few  may  not  be  an  easy 
matter.  Also,  assignments  are  made  and 
commitments  agreed  to  throughout  a  pro¬ 
ject.  But  in  Company  C’s  production  sys¬ 
tem,  firefighting,  frequent  block  points, 
and  resultant  overtime  caused  havoc  with 
the  CMMI  schedules. 

But  judicious  use  of  the  Swiss  Cheese 
Method  and  the  Pareto  Principle  com¬ 
bined  to  help  attain  full  CMMI  compli¬ 
ance  (Company  C’s  CMMI  appraisers 
called  it  validation)  prior  to  the  appraisal. 
Instead  of  juggling  three  or  four  balls, 
there  were  dozens  of  balls  in  the  air  at  any 
one  time. 

Lessons  Learned 


are  different  types  of  analysts,  devel¬ 
opers,  and  support  personnel.  The 
scope  of  the  PIT  also  evolved  and 
helped  focus  on  the  vital few  actions. 

•  IBM’s  Rational  ClearQuest  software 
was  used  for  change  requests  and 
Rational  ClearCase  for  deliverables 
(requirements,  design,  code,  etc.). 
Getting  these  under  configuration 
management  was  not  a  trivial  effort 
due  to  multiple  versions  on  file  servers 
and  MS  SharePoint 
Process  improvement  is  difficult  work, 
including  getting  ready  for  a  CMMI 
appraisal.  But  by  using  basic  time  manage¬ 
ment  techniques,  persistence,  and  creativi¬ 
ty,  organizations  can  maximize  their  prob¬ 
ability  of  success.  Lessons  learned  should 
be  documented  and  shared  with  others. 
As  the  saying  goes,  “We’re  all  in  this 
together.”^ 
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Company  C’s  experience  can  be  applied  to 
others  planning  a  CMMI  adventure.  Here 
are  some  other  lessons  learned: 

•  Since  onsite  process  workshops  weren’t 
feasible,  virtual  workshops  were  held. 
Instead  of  flip  charts  on  the  wall,  the 
participants  used  a  one -page  graphic 
called  the  project  process  flow.  This  com¬ 
plex  diagram  showed  every  step  of  the 
project  using  Project  Management 
Body  of  Knowledge  categories  such  as 
planning,  executing,  controlling,  and 
closing.  Company  C’s  dozens  of 
process  documents  (completed  and  in- 
process)  were  identified.  A  sub-group 
was  assigned  to  update  the  software 
development  life  cycle  processes 
(another  non-trivial  effort). 

•  Throughout  the  project,  a  CMMI 
appraiser  was  involved.  Periodic  valida¬ 
tions  were  performed  to  identify  areas 
in  compliance  and  those  needing 
either  improvement  or  time  to  become 
institutionalized. 

•  The  list  of  participants  in  the  appraisal 
constantly  evolved.  The  Process 
Improvement  Team  (PIT)  members 
negotiated  the  list  with  management  to 
help  assure  that  aU  job  categories  were 
adequately  represented. 

•  PIT  membership  was  also  frequently 
negotiated  with  management.  There 


p  ■ 


CIVILIAN  TALENT  IS  MISSION-CRITICAL. 

LET'S  GET  TO  WORK. 

N  AV^^Ta  I  R 

CIVILIAN 
CHoice  IB  Yours. 

Di&oovaj  mors  about  Naval  Air  Systems  Command  todoy. 

£aio  www.novoir.navy.mit 


Work  for  Naval  Air  Sysfems  Command  (NAVAIR)  and  you'll  support  our  Sailors 
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Web  Sites 


Information  Technology  Infrastructure 
Library  (ITIL) 

www.itil-ofFicialsite.com 

After  reading  Jeffrey  L.  Duttons  article,  explore  one  of  the  com¬ 
ponents  of  his  integrated  framework  to  process  improvement. 
This  official  Web  site  for  the  ITIL  offers  a  comprehensive  host  of 
resources  for  ITIL  users.  Learn  about  ITIL’s  benefits  in  regards  to 
costs,  capabilities,  and  customer  satisfaction;  read  testimonials, 
white  papers,  and  case  studies  from  users  and  organizations 
worldwide;  and  learn  everything  about  ITIL  certification  from 
training  to  exams  to  what  it  takes  to  earn  the  ITIL  Expert  Level. 
There  is  also  the  recent  “refresh”  from  ITIL  Version  2  to  Version 
3:  The  site  navigates  readers  through  the  differences  between  the 
two  versions,  discusses  Version  2’s  future,  and  outlines  the  bene¬ 
fits  of  Version  3. 

Webinar:  CMMI VI  .3  Product  Suite 

www.sei.cmu.edu/library/abstracts/webinars/10feb2009.cfm 
A  perfect  supplement  to  Mike  Phillips  and  Sandy  Shrum’s  Process 
Improvement  for  All:  What  to  Expect  from  CMMI  Version  1.3,  this 
webinar  session  discusses  CMMI  VI  .3.  Mike  Konrad,  manager  of 
the  CMMI  Model  Team  and  CMMI  Chief  Architect,  and  Rusty 
Young,  manager  of  the  SCAMPI  Appraisal  Team,  focus  on 
updates  to  the  suite:  clarity  of  high  maturity;  more  effective  gener¬ 
ic  practices;  appraisal  efficiencies;  and  commonality  across  the 
constellations  of  Development,  Acquisition,  and  Services. 

Information  and  Communications 
Technology  (ICT)  Project  Management 
Practitioner  Network 

http://ictpmpractitioner.ning.com/ 

After  reading  Carol  Dekkers  and  Pekka  Forselius’  article  on  ICT 
program  recovery,  you  may  want  to  join  the  free  online  learning 
network  designed  for  both  beginners  and  advanced  practitioners 
of  project  management  in  ICT  solutions  development.  Users 
freely  initiate  discussion  and  present  best  practices  on  how  to 
effectively  coordinate  ICT  initiatives  in  government,  education, 
people  organization,  and  small  and  medium  enterprises.  Users 
can  also  customize  their  own  pages  with  discussion  forums,  blogs, 
and  video  to  facilitate  what  the  ICT  Project  Management 
Practitioner  Network  calls  “the  open  online  discussion  and  shar¬ 
ing  of  the  practitioner’s  contextual  application  of  project  stan¬ 
dards  and  ICT  solutions  development.” 


continuous  improvement,  focusing  on  four  key  elements:  a 
broad-based  and  structured  Continuous  Process  Improvement 
implementation  method,  aligned  goals,  project  management  and 
implementation,  and  providing  a  framework  and  useful  checklists 
to  gauge  organizational  maturity  in  using  Continuous  Process 
Improvement. 

CMMI  Appraisals 

www.sei.cmu.edu/ cmmi/ tools/ appraisals 

Darrell  Corbin’s  CMMI,  Swiss  Cheese,  and  Pareto  explained  how 
to  prepare  for  an  CMMI  appraisal — now  go  the  SEI’s  official 
source  for  information  on  the  CMMI  Appraisal  Program.  The 
Web  site  details  the  Appraisal  Requirements  for  CMMI,  includ¬ 
ing  the  characteristics  of  SCAMPI  at  Class  Levels  A,  B,  and  C. 
There  are  also  resources  including  publications  and  presentations 
related  to  appraisals,  frequently  asked  questions,  official  commu¬ 
nications  directly  from  the  SEI’s  Appraisal  Program,  an  SEI 
Partner  directory  and  network,  and  a  link  to  published  appraisal 
results. 

CMMI  or  Agile:  Why  Not  Embrace  Both! 

www.sei.cmu.edu/repoits/08tn003.pdf 

As  an  exceptional  companion  piece  to  his  article  Love  and 
Marriage:  CMMI  and  Agile  Need  Each  Other,  Hillel  Glazer  (this 
time  with  co-authors  Jeff  Dalton,  David  Anderson,  Mike  Konrad, 
Sandy  Shrum)  explore  the  two  methods  offen  at  odds  with  each 
other.  Their  report  clarifies  why  the  discord  need  not  exist  and 
proposes  that  CMMI  and  Agile  champions  work  toward  deriving 
benefits  from  using  both  and  exploiting  synergies  that  have  the 
potential  to  dramatically  improve  business  performance.  The 
authors  outline  the  two  primary  reasons  for  the  discord,  examine 
the  divergent  origins  of  both  methods,  analyze  and  debunk  the 
factors  that  have  led  to  the  negative  perception,  detail  “truths” 
about  the  two  methods,  show  the  value  of  both  through  analysis 
and  comparison,  and  even  provide  a  “call  to  action”  for  both 
CMMI  and  Agile  experts. 

Crosstalk:  Looking  Back  at  CMMI 

www.stsc.hill.af  mil/ crosstalk/2007 /Q2/ 
www.stsc.hill.af  mil/ crosstalk/2006/ 04/ 
www.stsc.hill.af  mil/ crosstalk/2002/ 02/ 
www.stsc.hill.af  mil/ crosstalk/2000/ 07/ 


DoD  Continuous  Process  Improvement 
Transformation  Guidebook 

https://acc.dau.mil/CommunityBrowser.aspx?id=32364 
Ted  Warren  and  Ron  Abler  introduced  readers  to  a  “stealth” 
method  of  achieving  Continuous  Process  Improvement  through 
work  product  management,  now  explore  the  guide  used  by  the 
DoD  to  design  and  manage  Continuous  Process  Improvement 
efforts.  The  guide  standardizes  terminology  and  incorporates  best 
practices  from  the  experiences  of  both  defense  organizations  and 
leading  private  industries.  It  is  a  strategic  approach  for  developing 
a  culture  of  continuous  improvement  in  the  areas  of  reliability, 
process  cycle  times,  and  costs  in  terms  of  less  total  resource  con¬ 
sumption,  quality,  and  productivity.  The  guide  provides  a  frame¬ 
work  that  is  used  for  implementing  and  sustaining  a  culture  of 


If  you’re  new  to  CrossTalk,  you  may  have  missed  past  issues 
that  also  explored  the  famed  process  improvement  approach.  The 
February  2007  edition  explored  the  changes  in  Version  1.2, 
Appraisal  Method  for  Process  Improvement  B,  and  how  to 
achieve  a  CMMI  Level  2  rating  within  six  months.  You  can  also 
see  if  Watts  S.  Humphrey,  Dr.  Michael  D.  Konrad,  James  W. 
Over,  and  William  C.  Peterson  were  right  with  predictions  in 
their  article  Euture  Directions  in  Process  Improvement.  April  2006, 
themed  Alternate  Mixes  for  CMMI,  included  discussions  of  com¬ 
bining  Agile  and  traditional  software  development  techniques, 
managing  cultural  changes,  and  performing  cooperative 
appraisals.  Issues  dedicated  to  CMMI  also  reach  back  early  in  the 
decade,  with  February  2002  simply  themed  CMM/ and  July  2000 
focused  on  Process  Alchemy  in  CMMI. 
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Love  and  Marriage: 

CMMI  and  Agile  Need  Each  Other 


Hillel  Glazer 
Entinex,  Inc. 


Agile’s  values  and  practices  ensure  critical,  long-term  process  success,  making  it  an  ideal partner  of  the  CMMI'  framework, 
which  delivers  a  robust  infrastructure  of  organisation-wide,  broadly  inculcated  continuous  improvement  and  optimisation. 
Intended for  an  audience  of  process  improvement  professionals,  CMM  left  out  some  of  the  basic  elements  critical  to  long-term 
process  success  [1]  that — as  luck  would  have  it — Agile  values  and  practices  supply.  Together,  Agile  and  CMMI  complete 
each  others’  capabilities  and  can  lead  to  fast,  affordable,  visible,  and  long-term  benefits. 


In  early  2001,  the  Agile  movement  can 
be  arguably  said  to  have  gelled  with 
the  formulation  of  the  “Manifesto  for 
Agile  Software  Development”  (often 
called  the  Agile  Manifesto)  [2],  Other 
than  two  items  published  later  that 
year — one  in  CrossTalk  [3]  and 
another  in  IEEE  Software  [4] — much  of 
the  writing  from  2001-2008  on  the  topic 
of  CMM  [5,  6]  (or  CMMI)  and  Agile 
development  had  been  limited  to  online 
sources  such  as  e-mail  groups,  user 
forums,  blogs,  and  wikis^.  In  that  time, 
much  of  what  was  written  on  the  topic 
was  mostly  on  how  the  two  bodies  of 
ideas  were  incompatible. 

Barry  Boehm  and  Richard  Turner’s 
“Balancing  Agility  and  Discipline”  [7] 
provides  a  sound,  practical,  robust  risk- 
based  approach  to  reconciling  what  was 
widely  perceived  as  being  orthogonal 
interests  of  discipline,  a  la  CMMI  (then 
vl.l)  and  agility  (of  any  variety). 
Nonetheless,  the  process  myth  of  irrecon¬ 
cilability  between  CMMI  and  Agile  per¬ 
sisted.  User-group-type  organizations — 
such  as  the  SETs  Systems  and  Software 
Process  Improvement  Network  and 
Agile  Project  Leadership  Network 
(APLN)  groups — were  fostering  gather¬ 
ings  based  on  the  topic,  but  many  were 
billed  as  confrontational  panel  discussions 
and/ or  contrarian  viewpoints  [8] .  Over 
time,  however,  the  topic  began  to 
become  more  seriously  inspected,  fueled 
by  curiosity  as  described  in  [9,  10,  11]. 

There  have  been  various  missteps 
made  throughout  the  existence  of 
CMMI  and  its  predecessors  [12]  that 
contributed  to  the  creation  of  the  Agile 
Manifesto  [13].  Unfortunately,  these 
missteps  will  continue  to  be  made  by 
people  who  are  not  appropriately  quali¬ 
fied  to  be  using,  appraising,  or  teaching 
CMMI  (more  on  that  later).  Nonethe¬ 
less,  making  progress — in  a  world  that 
accepts  structured,  deliberate,  and  per¬ 
sistent  improvements  together  with 
empowered  teams.  Lean  processes. 


experimentation,  and  involved  cus¬ 
tomers — requires  that  all  parties: 

•  Understand  the  foundation  and 
intent  of  both  CMMI  and  Agile. 

•  Implement  CMMI  and  Agile  goals, 
values,  and  practices  in  synergistic 
ways. 

Both  Agile  and  CMMI  have  been 
shown  to  benefit  project  and  organiza¬ 
tional  performance.  To  gain  the  maxi- 


*^CN\N\\  focuses  on 
practices  and  artifacts 
of  cultures  of  process 
excellence  without 
addressing  the 
underlying  enablers 
of  this  culture/* 


mum  results  of  a  combined  approach, 
appropriate  expectations  from  both 
must  be  set.  Before  this  can  be  done,  we 
must  also  incorporate  particular  context 
of  CMMI  and  Agile. 

CMMI  Isn’t  for  Everyone — 

It’s  for  Experts 

Whether  intentional  or  not,  the  body  of 
work  that  is  the  CMMI  and  accompany¬ 
ing  products  and  services  (appraisals, 
training,  etc.)  in  large  part  targets  to  an 
audience  of  subject-matter  experts 
(SMEs):  people  with  knowledge,  train¬ 
ing,  skills,  and  a  foundation  of  process 
improvement  theory  and  practice.  There 
is  a  tacit  assumption  that  they  have 
broad,  practical,  applied  improvement 
experience  in  their  particular  domain  of 
work  and  are  professionals  in  process 
improvement.  For  process  improvement 


SMEs,  CMMI  is  abstract  enough,  but  for 
people  without  a  priori  background, 
experience,  and  education  in  process 
improvement,  CMMI  is  hard  to  use  and 
lacks  sufficient  context  and  background 
on  fundamental  process  improvement. 
CMMI,  perhaps  rightly  so,  doesn’t 
regress  to  explain  the  foundation  from 
which  it  emerged  [1]. 

CMMI  focuses  on  practices  and  arti¬ 
facts  of  cultures  of  process  excellence 
without  addressing  the  underlying 
enablers  of  this  culture.  In  fact,  the  term 
culture  in  any  form  appears  only  sparing¬ 
ly  in  CMMI:  twice  in  reference  to  choos¬ 
ing  a  model  representation  (which  only  real¬ 
ly  matters  when  pursuing  appraisals,  not 
improvements);  once  buried  in  an  exam¬ 
ple  within  a  subpractice  of  the 
Organizational  Innovation  and  Deploy¬ 
ment  process  area  (which  is  at  Maturity 
Level  5);  once  in  an  example  in  the 
introductory  notes  to  the  Process  and 
Product  Quality  Assurance  process  area; 
and  once  in  the  glossary  definition  of 
institutionalisation.  As  well,  there  is  no 
discussion  of  how  to  attain  a  culture 
conducive  to  process  improvement  or 
what  its  attributes  are. 

Although  built  on  decades  of 
process  improvement  practice — from 
Deming,  Juran,  and  Crosby  to  Ohno, 
Shingo,  and  the  work  at  Toyota’ — 
CMMI  doesn’t  mention  these  thought- 
leaders,  nor  does  CMMI  directly  explain 
that  their  work  in  creating  and  fostering 
high-performance  organizations  is  both 
context  to  and  reflected  in  CMMI. 
While  not  explicitly  prerequisites  to 
using  CMMI,  it  stands  to  reason  that 
having  the  basic  knowledge,  technique, 
and  application  of  process  improvement 
makes  success  with  using  CMMI  more 
achievable.  The  audience  for  CMMI  is 
assumed  to  have  a  working  knowledge 
of  and  background  in  basic  process 
improvement. 

Just  as  there  are  no  actual  processes 
in  CMMI — and  none  of  the  process 
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areas  have  enough  content  to  actually 
create  fully  functioning  processes  for 
actually  developing  real  products  and 
services — CMMI  similarly  doesn’t 
include  the  principles  and  practices  of 
basic  process  engineering  and  design. 
Again,  this  entry-level  content  isn’t 
something  one  would  expect  to  see  in  an 
advanced  work  on  process  improve¬ 
ment;  nonetheless,  when  people  start 
with  CMMI  without  this  context,  their 
failure  is  predictable. 

The  contents  of  the  People 
Capability  Maturity  Model  [14]  aren’t 
part  of  CMMI  either.  Organizations 
believe  they  can  achieve  high  value  in 
CMMI  implementations  by  focusing 
exclusively  on  CMMI  while  ignoring 
workforce  competency  and  empower¬ 
ment  facets  of  a  process  improvement 
culture  (which  happen  to  be  among  the 
Agile  principles).  The  attributes  and 
capabilities  of  high-performing,  strongly 
process-oriented,  and  well-integrated 
cultures  of  improvement  are  captured  in 
[15]. 

Agile  Limitations  (That 
CMMI  Can  Mitigate) 

AgHe  values,  principles,  and  practices 
have  been  demonstrably  beneficial  to 
many  organizations.  The  story  is  not  all 
rosy,  however,  and  there  are  plenty  of 
examples  of  Agile  failing  to  achieve 
desired  outcomes  just  as  there  are  fail¬ 
ures  of  CMMI  to  achieve  desired  out¬ 
comes'*.  Failures  with  Agile  often  have 
similar  causes  to  those  in  CMMI:  failing 
to  account  for  context,  background,  and 
culture,  and  implementing  incomplete 
components. 

However,  some  Agile  failures  can  be 
mitigated  with  the  practices  and  con¬ 
structs  in  CMMI.  Agile  values,  princi¬ 
ples,  and  practices  are  mostly  oriented  at 
the  team  and  project  level  and  their 
premises  rely  heavily  on  individuals  and 
those  currently  and  immediately 
involved  in  a  particular  effort.  CMMI 
goals  and  practices  assume  an  organiza¬ 
tion  wants  its  processes  propagated 
widely  and  over  distances  of  time 
and/ or  space.  Therefore,  CMMI  pro¬ 
vides  an  organizational-level  infrastruc¬ 
ture  as  well  as  mechanisms  to: 

•  Preserve  information  and  knowledge 
over  time. 

•  Structure  and  provide  criteria  for 
decision-making. 

•  Strengthen  and  normalize  risk  man¬ 
agement. 

•  Methodically  apply  technical  ap¬ 
proaches. 


•  Specifically  focus  on  process 
improvements. 

•  Specify  engineering  practices. 
(CMMI  includes  several  engineering 
best  practices,  but  assumes  users 
know  the  basics). 

Additionally,  CMMI  includes  prac¬ 
tices  for  the  normalization  of  processes 
at  the  organizational  and  project  levels 
that  are,  upon  closer  inspection,  attribut¬ 
es  of  an  organizational  culture  of 
process  performance.  These  generic  prac¬ 
tices  are  frequently  overlooked  by  organi¬ 
zations  using  Agile  approaches  since  the 
focus  of  Agile  values  is  too  often 
applied  in  a  short-term  and  target  effort 
view,  frequently  failing  to  take  into 
account  the  value  of  the  things  on  the  right 
[2,  13]. 


*^Failures  with  Agile 
often  have  similar  causes 
to  those  in  CMMI: 
failing  to  account  for 
context,  background, 
and  culture,  and 
implementing  incomplete 
components/* 


Another  area  where  Agile’s  content 
clearly  has  no  material  is  in  the  area  of 
quantitative  process  performance.  Some 
might  argue  that  there  is  no  point  to 
pursuing  quantitative  optimization  since 
the  very  notion  assumes  a  process  is  per¬ 
formed  with  enough  frequency  and  reg¬ 
ularity  that  pursuit  of  a  quantitative 
model  is  value-added. 

This  is  a  valid  argument,  especially 
for:  projects  that  never  have  anything  in 
common  with  other  projects;  projects 
that  are  short-term  and/ or  require  mini¬ 
mal  effort;  organizations  that  band  and 
disband  in  an  ad-hoc  fashion  as  a  func¬ 
tion  of  project  scope  and  client  need; 
and  organizations  that  don’t  understand 
that  quantitative  performance  need  not 
be  onerous  [16].  However,  for  projects 
involving  more  than  one  or  two  people, 
that  take  months,  that  have  teams  whose 
resource  pool  (of  a  dozen  or  so)  is  sta¬ 
ble  and  reusable,  and  for  organizations 
that  approach  development  valuing  cer¬ 
tain  aspects  of  performance  predictabil¬ 
ity,  process  optimization  is  neither  out- 
of-reach  nor  pointless.  In  fact,  processes 


can  be  described,  stabilized,  normalized, 
capable,  baselined,  and  optimized  with 
just  a  few  iterations  or  sprints. 

With  so  much  of  Agile’s  content 
drawing  from  Lean  sources,  it  is  con¬ 
spicuously  lacking  in  quantitative  tech¬ 
niques  that  are  so  much  of  a  staple  in 
those  very  same  practices.  Concepts 
such  as  TQM,  Six  Sigma,  and  the  Toyota 
Production  System  [17,  18,  19] — from 
which  several  highly  valued  Agile  princi¬ 
ples  are  based — are  all  more  than  princi¬ 
ple  and  culture  alone:  They  are  deeply 
quantitative  and  steeped  in  detailed 
process  definitions  and  standards.  Yes, 
the  culture  preceded  the  process  defini¬ 
tion  and  statistics,  but  the  process  defin¬ 
ition  and  statistics  are  also  a  reflection  of 
the  culture  as  well  as  a  facilitator  of  the 
relentless  pursuit  of  customer  delight. 
Many  practitioners  of  Agile  values  and 
principles  stop  well  short  of  quantitative 
techniques  and  process  definitions, 
thereby  making  an  unwitting  shortfall  in 
their  own  pursuits  of  excellence. 

This  is  another  role  played  by  misin¬ 
terpretation  of  CMMI.  Processes  (in  the 
large)  are  not  quantitatively  character¬ 
ized,  but  rather  the  focus  is  on  subprocess¬ 
es  [20].  Measures  are  taken  at  specific 
points,  not  at  all  process  junctions,  and 
not  throughout  a  process’  use,  but  where 
it  makes  business  sense  and  adds  organi¬ 
zational  value.  With  this  in  mind,  even 
Agile  organizations  will  find  that  there 
are  many  activities  they  perform  with 
regularity  even  when  projects,  teams, 
and  customers  are  different.  Some 
examples  are: 

•  Refactoring. 

•  Continuous  integration. 

•  Test-driven  development. 

•  Sprint /iteration  planning. 

•  Planning  Poker  estimation. 

•  Pair  programming. 

Each  of  these  practices  are  not  total 
processes;  they  are  made-up  of  sub¬ 
processes  which  can  be  measured  for 
duration,  defects,  effort,  instances 
(counts),  and  so  forth.  For  certain  appli¬ 
cations  within  certain  projects,  there 
may  be  benefits  to  normalizing  and  then 
quantifying  how  some  of  the  underlying 
subprocesses  in  these  practices  perform. 
Dismissing  quantification  out-of-hand  is 
nowhere  to  be  found  in  the  Agile  values 
or  practices. 

Agile  Teachings  (That  CMMI 
Can  Benefit  From) 

Simplicity.  Putting  the  basics  back  into 
the  improvements. 

Among  the  several  catalysts  for  the 
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Agile  movement  were  troubling  fads  in 
the  software  development  world  regard¬ 
ing  processes,  tools,  and  methodologies. 
In  particular,  these  fads  were  fast 
approaching  the  status  of  trends,  pulling 
the  development  world  into  a  vicious  cycle 
of  under-performing  project  results — 
despite  the  ever-increasing  presence  of 
concepts  ostensibly  created  to  bring 
about  success. 

The  emergence  of  the  Agile  move¬ 
ment  did  the  software  development 
world  (and  the  world  of  process 
improvement,  in  general)  a  great  service 
by  reminding  us  of  the  fact  that  individ¬ 
uals  and  interactions,  working  products, 
customer  collaboration,  and  responding 
to  change  do  matter  more  than  process¬ 
es  and  tools,  unnecessary  documenta¬ 
tion,  contract  negotiation,  and  continu¬ 
ing  to  follow  an  obsolete  plan  [16]. 
Specifically,  the  things  that  matter  most 
to  the  customer  are  the  things  the  cus¬ 
tomer  perceives,  experiences,  and  pays 
for.  In  other  words,  the  things  that  mat¬ 
ter  to  an  organization  are  facilitated  by 
delivering  on  things  that  matter  to  the 
customer.  Therefore,  customer  needs 
are  (and  always  have  been)  a  higher  pri¬ 
ority  to  the  viability  of  any  business 
transaction  than  the  unseen  machinations 
that  enable  the  organization  to  deliver 
against  those  customer  needs. 

Is  this  to  say  that  the  unseen  machi¬ 
nations  are  entirely  unimportant?  No. 
For  the  long-term  viability  of  the  busi¬ 
ness,  the  underlying  processes  that  make 
the  business  operate  are  critical. 
However,  these  are  less  important  to  the 
customer  than  is  meeting  their  immedi¬ 
ate  expectations.  A  simple  way  to  reiter¬ 
ate  [2]  in  a  process  improvement- orient¬ 
ed  way  is  to  say  all  efforts  must  align  with  the 
needs  of  the  business  to  satisfy  the  customer. 

Another  practice  found  among 
Agilistas  is  that  of  experimentation:  a 
time-honored  process  improvement 
technique  long  ago  lost  among  the  level- 
manic  set.  Not  just  experimenting  with 
solutions  in  the  fail  early  and  often  Agile 
sense,  but  experimenting  with  processes, 
organizations,  data  collection,  and  track¬ 
ing  techniques.  It’s  not  uncommon  to 
hear  “...  we  don’t  want  to  try  that 
because  it  might  kill  our  level.” 

Mired  in  the  unnecessary  bureaucra¬ 
cies  of  formal  process  groups,  the  idea 
of  experimentation  with  processes  has 
been  nearly  erased  from  the  process 
improvement  tool  kit.  In  CMMI,  the 
term  experiment  appears  exactly  three 
times,  not  one  of  which  is  connected  to 
basic  process  development.  The  closest 
reference  to  the  term  is  found  in  the 


Causal  Analysis  and  Resolution  process 
area  (which  most  organizations  don’t 
look  at  until  pursuing  Maturity  Level  5) 
as  a  suggestion  within  a  subpractice 
related  to  implementing  “action  propos¬ 
als”  [21]. 

Again,  this  is  not  a  dig  at  CMMI — it 
is  pointing  out  the  extent  of  context  and 
knowledge  assumed  to  exist  among 
CMMI  users.  If  any  criticism  of  CMMI 
in  this  regard  is  warranted,  it  is  that  the 
model  front-matter  does  not  highlight 
these  assumptions.  It  leaves  hapless  pro¬ 
ject  members  without  a  map  of  steps 
and  tools  to  navigate  by,  drowning  in  the 
gravity  well  of  process  improvement 
intricacies.  The  front-matter  states: 

The  audience  for  this  model 
includes  anyone  interested  in 
process  improvement  in  a  devel- 

danger  posed  to 
practitioners  of  both 
CMMI  and  Agile  is  the 
incorrect  assumption 
that  using  either  is  a 
substitute  or  an  excuse 
for  ignoring  the  discipline 
inherent  in  actual 
engineering  activities 


opment  and  maintenance  envi¬ 
ronment.  Whether  you  are  famil¬ 
iar  with  the  concept  of  capability 
maturity  models  or  whether  you 
are  seeking  information  to  get 
started  on  your  improvement 
efforts,  this  document  will  be 
useful  to  you. 

This  model  is  also  intended  for 
people  who  want  to  use  an 
appraisal  to  see  where  they  are, 
those  who  already  know  what 
they  want  to  improve,  and  those 
who  are  just  getting  started  and 
want  to  develop  a  general  under¬ 
standing  of  the  CMMI  for 
Development  constellation.  [22] 

Without  any  mention  of  assumed 
capabilities,  experience,  training,  apti¬ 
tudes,  knowledge,  or  skills,  CMMI  is 


foisted  by  level-hungry  management  on 
unsuspecting  users.  Instead,  readers 
most  likely  expected  sufficient  content 
in  the  model  explaining  not  only  what 
needs  to  be  done,  but  guiding  them  in 
even  the  most  macro-level  best  practices 
during  implementation.  Psychological 
change  management  skills  and  laying  the 
groundwork  for  the  right  culture  are 
critical  to  implementing  CMMI,  yet  are 
absent  from  the  text.  Organizations  left 
without  the  appropriate  understanding 
are  presumed  to  use  CMMI  not  knowing 
they  lack  the  wherewithal  to  get  any¬ 
where. 

But  thanks  to  Agile’s  values,  princi¬ 
ples,  and  practices  and  their  simple  por¬ 
trayal  of  the  basics  of  process  design 
and  use,  organizations  can  adopt  Agile 
ideas — and  with  them  implement 
CMMI — ^without  first  becoming  masters 
of  process  improvement.  Agile  can  pre¬ 
vent  the  bloating  of  processes,  ensure 
that  processes  add  value,  involve  the 
necessary  stakeholders,  encourage 
experimentation,  and  replace  level  mania 
with  results. 

The  Perfect  Marriage? 

Neither  CMMI  nor  Agile  include  con¬ 
tent  that  replaces  thorough  engineering 
practices.  A  danger  posed  to  practition¬ 
ers  of  both  CMMI  and  Agile  is  the 
incorrect  assumption  that  using  either  is 
a  substitute  or  an  excuse  for  ignoring  the 
discipline  inherent  in  actual  engineering 
activities  (or  appropriate  activities  for 
acquisition  and/or  services). 

For  example,  an  organization  that 
needs  to  be  taught  how  to  analyze 
requirements  to  ensure  they  are  neces¬ 
sary  and  sufficient,  or  that  believes  a 
design  can  be  fully  illuminated  from 
conversation  alone  (and  doesn’t  need 
description  or  revisiting),  isn’t  doing 
engineering  and  is  probably  not  ready 
for  CMMI.  Furthermore,  regardless  of 
whether  or  not  such  an  organization  is 
using  CMMI  or  whether  or  not  they  are 
implementing  Agile  practices,  it  isn’t 
actually  developing  in  the  engineering 
sense.  Such  organizations  may  be  pro¬ 
gramming  or  coding,  but  they  are  not 
developing.  Not  performing  engineering 
is  nothing  more  and  nothing  less  than 
not  performing  engineering.  Having  said 
that,  not  every  project  actually  requires 
engineering  to  be  done  at  the  project 
level.  Some  projects  are  merely  carrying 
out  the  final  touches  on  works  that  have 
already  been  engineered  long  ago 
and/ or  by  another  organization,  leaving 
the  more  mechanical  work  to  others. 
Sadly,  it  is  also  too  often  that  engineer- 
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ing  is  ignored  when  it  very  much  ought 
to  be  done.  This  may  point  to  other, 
deeper  issues  with  software  develop¬ 
ment  in  general,  as  raised  in  [23,  24,  25, 
26], 

Simply  put,  when  actual  engineering 
is  being  performed,  it  produces  the 
appropriate  artifacts.  Neither  CMMI 
nor  Agile  can  supply  enough  content  to 
cause  (or  make  necessary)  actual  engi¬ 
neering  practices. 

The  following  examples  describe  a 
few  ways  in  which  CMMI  and  Agile 
were  able  to  work  synergistically  during 
development  activities. 

In  one  case,  Agile  practices  (Scrum) 
were  already  in  place  and  operating 
effectively.  A  customer  of  the  organiza¬ 
tion  required  a  CMMI  Maturity  Level  2 
rating  to  be  compliant  with  their  con¬ 
tract.  Although  the  manager  of  software 
development  believed  there  would  be  a 
benefit  to  implementing  the  practices  all 
the  way  through  Maturity  Level  3,  exec¬ 
utive  management  preferred  the  lower 
expense  and  faster  results  of  pursuing  a 
Maturity  Level  2  rating.  Nonetheless,  in 
roughly  nine  months,  the  software  team 
of  approximately  a  dozen  cross-trained 
developers  and  a  few  specialists  had  sat¬ 


isfied  the  Maturity  Level  2  goals,  also 
(without  explicitly  trying)  satisfying  the 
goals  of  at  least  two  or  three  additional 
process  areas.  Their  approach  included: 

•  Using  Scrum  to  manage  the  process 
deployment  and  improvement  effort 
and  operating  the  process  engineer¬ 
ing  group  as  a  Scrum  team. 

•  Using  measures  and  metrics  that  were 
both  easy  to  obtain  and  relevant  indi¬ 
cators  of  process,  project,  organiza¬ 
tional,  and  product  performance. 

•  Integrating  CMMI  practices  into  their 
workflow,  including  expanding  Scrum 
practices  to  account  for  all  process 
activities  (regardless  of  whether  they 
were  tied  to  CMMI  or  not) . 
Leveraging  Scrum’s  product  and 
sprint  backlogs,  daily  stand-up  meet¬ 
ings,  and  end-of-iteration  retrospec¬ 
tives  for  conducting  activities  that 
improve  their  processes. 

Creating  a  simple  developer  hand¬ 
book  that  explains  how  product 
development  took  place  and  points 
its  users  to  all  the  process  assets  nec¬ 
essary  to  do  their  jobs. 

Creating  templates  and  checklists  for 
the  routine  aspects  of  product  devel¬ 
opment  and  project  governance. 


INCOSE  International  Symposium 
Chicago,  IL,  USA,  July  12-15,  2010 

Twentieth  Annual  International  Symposium  of  the 
International  Council  on  Systems  Engineering 

The  INCOSE  International 
Symposium  is  tlie  premier 
inlernational  forum  for  .Systems  T 
Engineering.  Participants  network, 

share  ideas,  knowledge  and  Inlorfiational^vmpdsium 

practices,  and  learn  more  about 
the  most  recent  innovations,  trends, 
experiences  and  i.ssues  in  Systems  Engineering. 

This  year  INCOSE  is  celebrating  20  years  since  the  first 
meeting  of  the  founders.  Paper  authors,  panelists  and 
tutorial  presenters  are  encouraged  to  address  ways  in 
which  Systems  Engineering  principles  and  perspectives 
are  performed  today  and  how  Systems  Engineering  may 
influence  our  future.  Topics  of  value  include  technology 
insertion,  process  improvements,  and  organizational 
governance  of  the  systems  we  make,  manage,  operate  and 
maintain  over  their  life  cycle,  to  the  benefit  of  mankind. 


www.incose.org/symp201 0 


•  Adding  discipline  and  standards  to 
areas  of  work  that  had  been  allowed 
to  be  performed  freestyle. 

•  Rotating  personnel  on  and  off  of 
the  process  group  to  broaden  the 
experience  base  and  keep  interest 
fresh. 

None  of  these  attributes  of  the 
client’s  approach  are  necessarily  unique 
to  Scrum/Agile  or  CMMI;  however,  the 
key  factor  in  the  client’s  easy  success 
was  that  they  began  with  and  applied 
Agile  values  and  practices  to  implement 
CMMI,  and  learned  through  CMMI  the 
benefits  of  having  certain  practices  of 
theirs  be  more  under  their  control. 

This  resulted  in  all  projects  using  a 
single  set  of  broad  practices  and  mea¬ 
sures,  increasing  the  predictability  of 
project  activities  and  the  ability  to 
demonstrate  progress  in  process  effec¬ 
tiveness.  The  teams  also  worked  more 
steadily  with  fewer  disruptions  by  build¬ 
ing  specific  touch  points  into  their  work¬ 
flow  The  team  also  learned  that  process 
artifacts  were  less  valuable  and  more 
disruptive  when  not  fully  designed. 
Though  the  discipline  that  some 
process  activities  afforded  was  benefi¬ 
cial,  the  team  learned  that  taking  the  easy 
way  out  actually  proved  to  be  more  dis¬ 
ruptive  than  had  they  created  more 
appropriate  tools. 

In  another  case,  an  organization  had 
been  using  many  practices  from 
Extreme  Programming  for  some  time 
and  had  experienced  dramatic  improve¬ 
ments  from  their  previous  approach.  As 
a  result,  they  found  themselves  in  the 
most  well-suited  position  to  be  lever¬ 
aged  for  pursuing  Maturity  Levels  4  and 
5.  Rather  than  burdening  themselves 
with  traditional  measurement  tech¬ 
niques  and  objectives,  the  development 
leader  was  able  to  identify  several  natur¬ 
al  measurement  points  throughout  their 
workflow.  Such  natural  measurement 
points  fell  into  a  few  general  areas: 

•  Between  physical  steps  in  the  work¬ 
flow 

•  Wherever  automation  is  able  to  col¬ 
lect  data. 

•  Whenever  data  is  being  entered  into 
or  manipulated  in  a  tool. 

•  During  (or  subsequent  to)  refactor¬ 
ing  activities. 

•  At  the  beginning  of  iterations  where 
analysis,  estimates,  and  task  alloca¬ 
tion  is  performed. 

•  At  the  end  of  iterations  (and  at 
releases)  where  much  of  the 
progress-to-date  is  being  reviewed. 
One  interesting  decision  was  to  not 

attempt  certain  metrics  from  the  activi- 
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ties  of  pair  programming.  This  decision 
was  reached  when  the  impact  of  the 
measurement  effort  was  found  to 
exceed  the  value  of  the  measures.  The 
ability  to  describe  the  processes  affect¬ 
ing  the  measures  was  also  determined  to 
be  too  complex  and  therefore  onerous 
to  control.  Despite  enormous  pressure 
(and  opportunity)  to  gain  valuable  data 
to  facilitate  high-maturity  behaviors,  a 
business  decision  was  made  regarding 
which  processes  would  add  value  by 
being  optimized  (after  determining  that 
it  would  not  add  business  value  to 
attempt  to  optimize  pair  programming). 

Although  this  organization  has  yet 
to  attain  the  necessary  performance  and 
depth  of  measures  to  approach 
Maturity  Levels  4  and  5,  the  lessons  they 
learned  and  applied  from  both  Agile 
and  CAIMI  are  moving  them  towards 
becoming  a  more  highly  performing 
Agile  team.  One  lesson  that  has  been 
implemented  was  the  realization  that 
greater  discipline  and  finer  granularity 
in  backlog  management  leads  to  more 
accurate  estimates  of  task  effort. 
Another  lesson  was  that  the  intent  of 
CMMI  practices  may  already  be  accom¬ 
plished  by  existing  activities  when  such 
activities  are  viewed  in  broader  engi¬ 
neering  terms  in  addition  to  their  more 
common  Agile  terms. 

Conclusion 

CMMI  can’t  be  everything  to  all  users. 
Some  users  will  work  with  CMMI  from 
the  perspective  of  already  being  process 
improvement  experts  and  some  will  be 
novices.  Regardless,  it  is  easy  to  take 
wrong  turns  with  CMMI.  However 
CMMI,  in  the  right  hands,  can  facilitate 
an  evolutionary  path  towards  optimiza¬ 
tion. 

Agile  helps  improve  many  opera¬ 
tional  and  transactional  activities  but 
wasn’t  intended  to  provide  higher  levels 
of  organizational  constructs  to  facilitate 
long-term  process  evolution.  Nonethe¬ 
less,  Agile  can  jump-start  effective 
process  design  and  deployment,  and 
foster  a  culture  of  process  excellence 
through  its  core  values  in  Lean  and 
cooperative  processes. 

Together,  CMMI  and  Agile  can 
operate  synergistically  to  enhance  the 
other’s  performance,  speed  to  deploy¬ 
ment,  and  acculturation.  Organizations 
would  be  well-advised  to  set  aside  their 
prior  perceptions  of  CMMI  and  Agile 
compatibility  and  embrace  both  their 
mutually  beneficial  and  shared  vision: 
delivering  a  high-quality  product  to  the 
customer  on  time.^ 
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Software  Processes?  How  Bohring! 


After  12  years  in  the  software  industry,  I  recently  returned  to 
teaching.  I  enjoy  it  immensely,  and  especially  enjoy  teaching 
the  “Intro  to  Computer  Science  Principles.”  I  find  it  refreshing  to 
give  students  their  first  taste  of  computer  science.  I  am  always 
amused  by  their  initial  first  attempts  at  programming.  Just  like 
watching  a  newborn  calf  take  its  first 
teetering  steps,  watching  a  “new 
mind”  learn  to  write  programs  is 
both  rewarding  and  entertaining. 

In  my  graduate  courses,  however, 

I  expect  a  somewhat  higher  standard. 

I  am  pretty  strict  about  expecting  my 
students  to  follow  “good”  program¬ 
ming  practices — good  documenta¬ 
tion,  readable  code,  etc.  My  standards 
are  pretty  high,  and  once  the  students 
learn  what  I  expect,  they  eventually 
see  how  clear,  easy-to-read  code 
makes  debugging  and  maintenance 
easier  (either  that,  or  they  humor  me 
until  the  end  of  the  semester,  after 
grades  have  been  assigned). 

In  a  recent  grad  class,  I  was  lec¬ 
turing  on  some  subtle  point  of  oper¬ 
ating  systems.  I  was  using  Unix  as  an 
example,  and  I  had  pulled  up  an 
example  of  Unix  network  code  from 
a  handy  reference  manual.  As  my 
class  was  going  over  the  example, 
several  of  the  students  were  shocked 
by  some  of  the  “poor”  code  used  in 
the  operating  system.  Poorly  docu¬ 
mented.  Bad  variable  names. 

Heavens,  even  global  variables!  A 
couple  of  the  students  pointed  out 
that  I  would  have  quickly  given  them 
a  grade  of  zero  for  writing  code  like 
that. 

My  response  to  them  involved  the 
International  Obfuscated  C  Code 
Contest  (lOCCC).  The  purpose  of 
this  contest  is  to  award  to  most  cre¬ 
atively  obfuscated  code  (where 
obfuscated  means  to  make  things  as 
confusing  or  difficult  to  understand 
as  possible).  A  quick  example,  in  the 
sidebar,  is  the  code  that  will  generate 
the  entire  12  verses  of  The  12  Days  of  Christmas.  No  mis¬ 
prints — I  compiled  and  checked  it  to  make  sure!  See 
<http://en.wikipedia.org/wiki/Obfuscated_code>.  At  first 
glance,  it  looks  like  the  random  typing  of  a  drunken  cat  as  it  trods 
across  the  keyboard. 

Why  do  we  celebrate  poorly  written  code?  Mostly,  of  course, 
for  the  humor.  According  to  the  lOCCC  Web  site 
<wwwl.us.ioccc.org/main.html>,  it  “shows  the  importance  of 
programming  style,  in  an  ironic  way.” 

In  the  lOCCC,  I  am  sure  that  every  one  of  the  entries  was 
written  by  an  expert  coder.  They  already  know  how  to  write  very 


good  code — and  this  contest  is  a  safe  forum  poking  fun  at  them¬ 
selves.  If  my  students  wrote  code  Hke  this?  First  of  aU,  they 
would  never  be  able  to  debug  it,  so  grading  would  be  much  sim¬ 
pler.  And  I  could  always  hope  for  a  hammer  to  spring  from  a  key¬ 
board  and  crack  a  knuckle  if  a  student  tries  to  write  such  code.  I 
teach  my  students  to  follow  the  rules, 
because  it  allows  me  to  hold  them  to  a 
standard.  However,  an  experienced 
programmer  who  understands  the  sys¬ 
tem  can  safely  break  a  few  rules — as 
long  as  their  experience  allows  them  to 
do  it  safely.  There  are  exceptions  to 
every  rule,  and  an  expert  knows  when 
it’s  safe  to  break  a  rule.  But  you  have  to 
understand  the  processes  before  you 
start  breaking  them. 

One  reason  I  love  the  CMM/CMMI 
is  the  emphasis  on  “repeatable  process¬ 
es.”  A  long  time  ago,  I  had  a  friend  try 
and  teach  me  how  to  golf  He  said  I 
was  a  perfect  student — I  consistently 
made  the  same  mistakes  over  and  over 
(and  over  and  ...).  It’s  easy  to  fix  a  prob¬ 
lem  when  the  problem  is  repeatable. 

When  designing  software,  you  need 
to  be  able  to  understand  a  flawed 
process — and  make  it  better.  In  soft¬ 
ware  engineering,  I  teach  about  heisen- 
bugs  (named  after  the  Heisenberg 
Uncertainly  Principle)  and  bohrbugs 
(named  after  the  well-defined  Bohr 
atom  model)  which  are  bugs  that  are 
respectively  difficult  to  reproduce  and 
easy  to  reproduce.  A  heisenbug  can’t  be 
reliably  duplicated,  so  it’s  very  hard  to 
find  and  fix.  A  bohrbug,  however,  can 
be  reliably  reproduced,  making  it  some¬ 
what  easy  (or  at  least  easier)  to  trace 
and  fix. 

A  process  model  allows  you  to  cre¬ 
ate  bohrbugs  rather  than  heisenbugs. 
While  I’m  not  saying  it  is  perfectly  OK 
to  create  mistakes,  I  AM  saying  that  if 
you  want  to  improve  your  process,  you 
have  to  at  least  screw  up  in  a  repeatable, 
predictable  manner.  And  be  willing  to 
improve.  And  promise  to  NEVER 
write  code  like  the  12  Days  program. 

— David  A.  Cook,  Ph.D. 

Stephen  F.  Austin  State  University 
cookda@sfasu.edu 


How  Not  to  Write  Code 
1 2days.c 

#include  <stdio.h> 
main(t,_,a)char 

*a;  {return!0<t?t<3?main(-79,- 

1 3,a+main(-87,l  -_, 

main(- 

86,0,a+ 1) +a)):  1  ,t<_?main(t+ 1  ,_,a)  :3,mai 
n(-94,-27+t,a)&&t^^2?_<13? 
main(2,_+l,”%s  %d  %d\n”):9:16:t<0?t<- 
72?main(_,t, 

“@n’+,#’/*{}w+/w#cdnr/ +,{}r/*de} +,/*{*+ 
,/w{%+,/w#q#n+,/ #  {l,+,/n{n+,/ +#n+,/ #\ 
;#q#n+,/+k#;*+,/’r  :’d*’3,}{w+K 
w’K:’+}e#’;dq#’l  \ 

q#’+d’K#!/+k#;q#’r}eKK#}w’r}eKK{nl]’/#; 
#q#n’){)#}w’){){nl]’/+#n’;d}rw’  i;#  \ 
){nl]!/n{n#’;  r{#w’r  nc{nl]’/#{l,+’K 
{rw’  iK{;[{nl]’/w#q#n’wk  nw’  \ 
iwk{Kd<{nl]!/w{%’l##w#’  i; 

:{nl]’/*{q#’ld;r’}  {nlwb!/*de}’c  \ 

;;{nl’- 

{}rw]’/+,}##’*}#nc,’,#nw]’/+kd’+e}+;#’r 
dq#w!  nr’/  J  }-)-}{rl#’{n’‘)#  \ 

}’+}##(!!/”) 

:t<-50?_——*a?putchar(31[a]):main(- 
65,_,a+l):main((*a— — ’/’)+t,_,a+l) 
:0<t?main(2,2,”%s”);*a— =’/’  |  |  main(0,mai 
n(-61,*a, 

“!ek;dc  i@bK’(q)- 
[w]*%n+r3#l,  {  }  :\nuwloca-0;m 
.vpbks,fxntdCeghiry”),a+ 1);}  .vpbks,fxnt 
dCeghiry”),a+l);} 
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Building  Solutions  for  the  Systems 
of  the  Past,  Present,  and  Future! 


AS9100 


ISO  9001 


Please  contact  us  today 

Ogden  Air  Logistics  Center 
309th  Software  Maintenance  Group 
(formerly  MAS  Software  Maintenance  Division) 
Hill  Air  Force  Base,  Utah  84056 

Commercial:  (801)  777-2615,  DSN  777-2615 
E-mail:  ooalc.masinfo@hill.af.mil 
or  visit  our  website:  mnv.mas.hill.af.mil 
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Crosstalk  thanks 
the  above 
organizations  for 
providing  their  support. 


